{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# TensorFlow Tutorial #17\n",
    "# Estimator API\n",
    "\n",
    "by [Magnus Erik Hvass Pedersen](http://www.hvass-labs.org/)/[GitHub中文](https://github.com/Hvass-Labs/TensorFlow-Tutorials-Chinese)\n",
    "/ [GitHub](https://github.com/Hvass-Labs/TensorFlow-Tutorials) / [Videos on YouTube](https://www.youtube.com/playlist?list=PL9Hr9sNUjfsmEu1ZniY0XpHSzl5uihcXZ)\n",
    "\n",
    "中文翻译[ZhouGeorge](https://github.com/ZhouGeorge)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Introduction介绍\n",
    "\n",
    "在软件开发中高级API是相当重要的，因为它们可以为完成复杂的任务提供简单的方式。这样可以更简单地去完成和理解你的源码并且这样出错的风险很低。\n",
    "\n",
    "在教程 #03中我们见过如何用不同的构筑API在Tensorflow中创建神经网络。然而，为了训练模型或在新的数据上使用它们，需要添加很多的代码。 Estimator是另一个高级的API可以完成其中的大部分工作，尽管它易用性仍然是有待讨论的。\n",
    "\n",
    "使用 Estimator API 包括以下几个步骤：\n",
    "1. 给 Estimator 定义输入数据的函数。\n",
    "2. 或者使用一个已经存在的Estimator（例如 一个深度神经网络），称之诶为预制（pre-made）或封装（Canned）的Estimator。或者创建一个你自己的Estimator，你还需要定义优化器，评价指标，等等。\n",
    "3. 利用步骤1.中定义的数据集训练Estimator。\n",
    "4. 在步骤1.定义的测试机上评价Estimator的表现。\n",
    "5. 用训练好的Estimator去预测其他的数据。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 引导"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/magnus/anaconda3/envs/tf-gpu/lib/python3.6/site-packages/h5py/__init__.py:36: FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`.\n",
      "  from ._conv import register_converters as _register_converters\n"
     ]
    }
   ],
   "source": [
    "%matplotlib inline\n",
    "import matplotlib.pyplot as plt\n",
    "import tensorflow as tf\n",
    "import numpy as np"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "开发环境Python 3.6（Anaconda）和TensorFlow version："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'1.9.0'"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tf.__version__"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 加载数据"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "MNIST 数据集大学12MB，如果指定路径下不存在它会被自动下载。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "from mnist import MNIST\n",
    "data = MNIST(data_dir=\"data/MNIST/\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "MNIST数据集已经被加载了，它包含70000张图像和对应的标签（图像的分类）。数据集被划为3个子集。我们在这个教程中只需要用到训练集和测试集。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Size of:\n",
      "- Training-set:\t\t55000\n",
      "- Validation-set:\t5000\n",
      "- Test-set:\t\t10000\n"
     ]
    }
   ],
   "source": [
    "print(\"Size of:\")\n",
    "print(\"- Training-set:\\t\\t{}\".format(data.num_train))\n",
    "print(\"- Validation-set:\\t{}\".format(data.num_val))\n",
    "print(\"- Test-set:\\t\\t{}\".format(data.num_test))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "为了方便起见，复制一些数据的维度。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "# The number of pixels in each dimension of an image.\n",
    "img_size = data.img_size\n",
    "\n",
    "# The images are stored in one-dimensional arrays of this length.\n",
    "img_size_flat = data.img_size_flat\n",
    "\n",
    "# Tuple with height and width of images used to reshape arrays.\n",
    "img_shape = data.img_shape\n",
    "\n",
    "# Number of classes, one class for each of 10 digits.\n",
    "num_classes = data.num_classes\n",
    "\n",
    "# Number of colour channels for the images: 1 channel for gray-scale.\n",
    "num_channels = data.num_channels"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 画图的辅助函数"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "在3x3的网格中画9张图，包括在每张图下面显示正确的类别和预测的类别。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "def plot_images(images, cls_true, cls_pred=None):\n",
    "    assert len(images) == len(cls_true) == 9\n",
    "    \n",
    "    # Create figure with 3x3 sub-plots.\n",
    "    fig, axes = plt.subplots(3, 3)\n",
    "    fig.subplots_adjust(hspace=0.3, wspace=0.3)\n",
    "\n",
    "    for i, ax in enumerate(axes.flat):\n",
    "        # Plot image.\n",
    "        ax.imshow(images[i].reshape(img_shape), cmap='binary')\n",
    "\n",
    "        # Show true and predicted classes.\n",
    "        if cls_pred is None:\n",
    "            xlabel = \"True: {0}\".format(cls_true[i])\n",
    "        else:\n",
    "            xlabel = \"True: {0}, Pred: {1}\".format(cls_true[i], cls_pred[i])\n",
    "\n",
    "        # Show the classes as the label on the x-axis.\n",
    "        ax.set_xlabel(xlabel)\n",
    "        \n",
    "        # Remove ticks from the plot.\n",
    "        ax.set_xticks([])\n",
    "        ax.set_yticks([])\n",
    "    \n",
    "    # Ensure the plot is shown correctly with multiple plots\n",
    "    # in a single Notebook cell.\n",
    "    plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 画一些图像检验数据是否正确"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAUMAAAD5CAYAAAC9FVegAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAHihJREFUeJzt3XmUFNXZx/HvA0LYVQQFFWdOwAVCFBWDu0aBKCogccG4EGM0osEtAaNx1xglKBzRE7YD4QQNigKCUVFAEV8EJIIi4wYiCsRlhLggIsJ9/5i5XdUzPXtXVU/7+5zjmequ6qpnvPSdp27dxZxziIj80DVIOgARkVygylBEBFWGIiKAKkMREUCVoYgIoMpQRARQZSgiAqgyFBEBVBmKiACwS00ObtOmjSssLIwolNzzwQcfUFxcbEnHESeVcf5TGWdWo8qwsLCQZcuW1T6qeqZ79+5JhxA7lXH+UxlnpttkERFUGYqIAKoMRUQAVYYiIoAqQxERoIZPk0Vqa8SIEQBs3boVgDfeeAOAxx9/vNyxgwcPBuCoo44C4MILL4wjRPmBU2YoIoIyQ4nYueeeC8C0adMy7jcr3xd2zJgxAMydOxeAE044AYD99tsvihAlQe+++y4ABx54IAAPPPAAAEOGDIk9FmWGIiIoM5QI+GwQKs4IDzroIABOOeUUAN5///3UvlmzZgGwevVqAKZMmQLAjTfemP1gJVHLly8HoEGDkrxsn332SSwWZYYiIigzlCzy411nzJhRbl/Xrl2BIOtr06YNAC1atADgu+++Sx3bo0cPAF5//XUAPv/884gilqStWLECCP4dDBgwILFYlBmKiBBDZuj7kY0fPx6AvffeO7WvSZMmAJx//vkAtGvXDoBOnTpFHZZE4L///S8AzrnUez4jnDNnDgDt27fP+FnfDxHgrbfeStt3+umnZzVOSd7KlSsBGD16NAAXXXRRkuEAygxFRIAYMsOhQ4cCJRMsVsT3K2vVqhUAXbp0ycq1O3ToAMCwYcOAH+bcdXE644wzgOApMEDLli0BaN26daWfffTRR1Pb4fZDyU/vvPMOAFu2bAHSeyAkRZmhiAiqDEVEgBhukydMmAAE3STCt8BFRUVA0PHyxRdfBGDx4sVAMPzqww8/rPD8jRo1AoKuGr4RP3wef7us2+R4FBQUVPvYv/3tb0AwLCvMd7HxPyV/DB8+HChZggBy47upzFBEhBgyw5NPPjntZ5gfiuVt3rwZCDJF/9fi1VdfrfD8P/rRj4BgoLcf5gWwadMmADp27Fir2CU6Tz31FAC33HILANu2bUvt22uvvQC45557AGjWrFnM0UkUwg9R/Xfaf2+bN2+eREhplBmKiJBjw/F23313AE466aS09zNllWU98cQTQJBdAhx88MEADBw4MFshSpb4oXvhjNDz3Sz81F2SHxYsWFDuvbZt2yYQSWbKDEVEyLHMsDY+/fRTAK644gogfSiYb4+qqsOvxKd///5AMDzPGzRoUGr7rrvuijUmiYdf6iHMD4jIBcoMRUTIg8zwoYceAoIMcbfddkvt80+qJHm+/+eiRYuAoK3QtxnddNNNqWP9dE6SH1555RUAJk2alHrv0EMPBaBXr16JxJSJMkMREepxZvjyyy8DQV8078knn0xt++mjJHl+0s7i4uK09/30beoLmr/mzZsHpPf08H2M/TR+uUCZoYgIqgxFRIB6fJv89NNPA8Hcdz179gTgqKOOSiwmKc+veeKHWHonnngiAHfccUfcIUnM/CQtYWeffXYCkVROmaGICPUwM9y6dSsAzz77LBBM1HD77bcDwZRekpzwanZ33303UH726m7dugHqRpPPPv74YwAWLlwIpE+icuaZZyYSU2WUGYqIUA8zQz8ZqG+DOvXUUwE4+uijE4tJ0t13332p7aVLl6bt88Px1FaY//7xj38A8MknnwDBdzVXKTMUEaGeZIZ+IlCAO++8E4Bdd90VgJtvvjmRmKRi999/f4X7/PBJtRXmv3Xr1qW99lP05SplhiIi5Hhm6J9KXnXVVan3vv/+ewD69OkDqF9hfePLtDpP/X3274/dvn07AF988UW5Y/1Qr5EjR2Y8V8OGDVPb9957L6DlBKI2e/bstNenn356QpFUjzJDERFUGYqIADl6m7xjxw4gmNli7dq1qX2dOnUCggcpUr/4dWmq45xzzgGgffv2QNBFY+rUqXWKwa++F55DUbLHd7L25VVfKDMUESFHM8M1a9YAwQpqYb7bhua/y13+4RbAzJkza32exx57rMpj/MOVBg3S/6737dsXCNbeDjv22GNrHZNUbcaMGUDwsNPPap3rqx0qMxQRIccyQ99Js3fv3mnvjxgxIrWd64/nBaZPn57aHj58OFB+ogavqKgIqLwd8JJLLgGgoKCg3L5f/vKXAHTu3Ll2wUrWfPPNNwA888wzae/76brC3ZtykTJDERFyLDMcO3YsUH4YT7itwcxijUnqprrr4j7yyCMRRyJR8+23foXKfv36AXD11VcnFlNNKDMUESFHMkPfL+nBBx9MOBIRqS2fGfp1kusbZYYiIuRIZujXQP7qq6/S3vejTTTdk4hETZmhiAiqDEVEgBy5TS7Lr5w2b948AFq3bp1kOCLyA6DMUESEHMkMb7jhhrSfIiJxU2YoIgKYc676B5t9Bqyr8sD8UeCca5t0EHFSGec/lXFmNaoMRUTylW6TRURQZSgiAkT8NNnM9gDmlb5sB+wAPit9/TPnXOYZP+t2zS5AeD6ojsANzjnNAhGBhMq4AJgM7Ak44O8q3+gkUcal150M9AE2OOe6RXGNtOvF1WZoZrcBXzvnRpR530rj2BnBNRsBG4DDnHPrs31+SRdXGZvZ3sCezrkVZtYKWA6c6px7Nxvnl4rF+T02sxOArcC4OCrDRG6TzayTmRWZ2cPAKqCDmf0vtH+gmU0o3d7LzKab2TIzW2pmR9bgUr2At1QRxi/KMnbObXTOrSjd/hJ4G9gnut9GMon6e+ycWwBsiuwXKCPJNsODgJHOuS6UZG8VeQAY7pzrDpwD+P+5PcxsTBXXGAj8KxvBSq1EXsZm9mOgK/BqdkKWGorjexyLJEegrHHOlV8LtLyewIGh6f53N7OmzrklwJKKPmRmTYDTgOvqHKnUVtRl3Ap4AhjinPu6ztFKbURaxnFKsjLcEtreCYQXN2kS2jZq10h7GrDEOVdcy/ik7iIrYzNrDEwHJjnnZtUpSqmLqL/HscmJrjWlja6bzWx/M2sAnBnaPRe40r8ws+o2pJ6HbpFzRjbLuLSx/h/ACufcAxGEK7UQ0fc4NjlRGZa6HpgDLALCDzyuBI4xszfMrAi4FCpvazCzlsDPgZnRhiw1lK0yPoGSP3a9zGxF6X+/iDh2qZ5sfo+nAQuBLma23sx+HWXgGo4nIkJuZYYiIolRZSgigipDERFAlaGICKDKUEQEqGGn6zZt2rjCwsKIQsk9H3zwAcXFxVb1kflDZZz/VMaZ1agyLCwsZNmy6oy8yQ/du3dPOoTYqYzzn8o4M90mi4igylBEBFBlKCICqDIUEQFUGYqIAKoMRUSAZCd3rdCWLSXzRQ4dOhSAMWOCGX78Y/Jp06YBUFBQEHN0IpKPlBmKiJCjmeHGjRsBGD9+PAANGzZM7fOdRWfPng3A73//+5ijk9p47bXXABgwYABQMiqgtp577rnUdufOnQHo0KFD7YOTxPjvcd++fQEYPXo0AIMHD04dE/7+R0mZoYgIOZYZfvbZZwAMGjQo4Ugk2+bMmQPAtm3b6nyuWbOC9Z8mTpwIwNSpU+t8XonP559/DqRngABDhgwB4JJLLkm917Rp01hiUmYoIkKOZIYPPFCywNnMmSXrN736atXrgS9cuBAAv4bLIYccAsDxxx8fRYhSS99//z0ATz/9dNbOGR54f//99wNBD4TmzZtn7ToSnZdeegmADRvS150/77zzAGjSpEm5z0RNmaGICDmSGV5zzTVAzZ4aTZ8+Pe3nfvvtB8Bjjz2WOubwww/PVohSSy+88AIAixYtAuD666+v8zk3bdqU2l61ahUA33zzDaDMMJeF24vvuuuujMdceOGFAJQsjR0vZYYiIqgyFBEBEr5N7tOnDxA8BNmxY0eVn2nTpg0Q3A6tW7cOgLVr1wJwxBFHpI7duXNn9oKValu5cmVqe+DAgQB06tQJgBtvvLHO5w93rZH644033kht+0743i67lFRFp556aqwxhSkzFBEhgcxwwYIFqe23334bCBpLK3qAcvnll6e2e/fuDcCuu+4KwPz58wH4y1/+Uu5zf//734HyHTslWuGy8A82pkyZAkCLFi1qfV7/4CT8byiJhnapHf+wM5NevXrFGElmygxFRIgxM/QD830bEkBxcXHGY303mbPOOguAW2+9NbWvWbNmacf6KbzGjh1b7pzDhg0D4NtvvwWCSR0aNWpUu19CKvX4448D6R2sfVthuC23tnx3jHA2eOKJJwKw22671fn8Eq1wRu81btwYgLvvvjvucMpRZigiQoyZ4fbt24GKs0EIhtI9+uijQPDkuDI+M/RPKa+77rrUPj9Ey2eIfpqgjh071ih2qR4/4a7//w7Zaa/1dxWPPPIIEDx5BLjpppsAZfu5zHe4f+WVV8rt83d63bp1izWmTJQZioiQI8PxfHvSpEmTgOplhGX5rO/hhx9Ovbd06dIsRCdV+eKLLwBYvHhxuX1XXHFFnc8/btw4IJjirUuXLql9J510Up3PL9GqbOKVXOrpocxQRIQEMsNMo0yWLFlS5/P6USzhUSdlR7b4p9K+z5tkhx+Av379eiCYhilb1qxZk/a6a9euWT2/RCtTZuif/mfjziFblBmKiKDKUEQEiPE22a99HNVKV36VreXLl6feKzvM7/bbb4/k2j90LVu2BILuEeGJGvwQutatW9f4vJ9++ikQdNnxjjnmmFrFKfF6+eWXgaBLVJgfTrvvvvvGGlNllBmKiBBjZvjUU09l9Xy+m0VRURFQ+XAe31VHHXOj4Vcv80Pv/LA8gNNOOw1I7wyfyZtvvpna9g9M/PRsZSdjaNBAf8PrA78Cnn+QGZYLEzOUpX9VIiLkSKfr2vDTRD300EMVHlNYWAjA5MmTgWACCInGbbfdBqRnAv6OIDxBRyZt27ZNbftMsKKhmxdffHFdwpSYlG3rDU+mcdlll8UdTpWUGYqIUA8zQ79UgJ8YtjJ+2NZxxx0XaUxSonPnzkD6CoX+6X7ZjtNl+enawgYNGgSU7yTv2yglN/nO92WfIoefHGdjSrdsU2YoIkKMmWFliz4988wzaa8vvfRSADZu3Fjheaoz3Xu2n2BLzR166KFpP2vixz/+ccb3w/0Yf/rTn9YuMImMn7Kr7FPkfv36JRFOtSkzFBFBlaGICBDjbbKft8zPOh3mO+aWHaqXaeiev82uzkp6Ur/526yyt1u6Nc5tvrO15wc9XHPNNUmEU23KDEVEiDEzHDBgAADDhw9PvVfZeihV8X9tfHeO8ePHA9C+fftan1Nyi39IprWR65c5c+akve7QoQMQTM6Qq5QZiogQY2boV7HzK98BzJw5E4BRo0bV+Hx//vOfgWAtZMk/fr1rT52tc5tfAXP16tVp7zdp0gTI/YlSlBmKiJDAcDy/NnJ4u3fv3kCwCpqfqPWMM84A4He/+13qM/7JYniFNMlPfrVEP8D/lltuSTIcqYKfWs0PtVu1ahUA+++/f2Ix1YQyQxERcmSihlNOOSXtpwgEGca1114LaI3kXOf7/vrp9XwvgMMOOyyxmGpCmaGICDmSGYpk4tuOpX7Ze++9AZg4cWLCkdSMMkMREVQZiogAqgxFRABVhiIigCpDERFAlaGICACWabX7Cg82+wxYF104OafAOde26sPyh8o4/6mMM6tRZSgikq90mywigipDERFAlaGICBDx2GQz2wOYV/qyHbAD+Kz09c+cc99FdN0+wEigITDWOfe3KK4jyZVx6bV3AV4D3nfO9Y/qOj90CX6PJwN9gA3OuW5RXCPtenE9QDGz24CvnXMjyrxvpXHszNJ1GgHvAD8HPgaWAb90zr2bjfNLxeIq49B5hwHdgGaqDOMRZxmb2QnAVmBcHJVhIrfJZtbJzIrM7GFgFdDBzP4X2j/QzCaUbu9lZtPNbJmZLTWzI6s4/ZHAW865dc65bcBjQL+ofhfJLOIyxswKgF7ApKh+B6lc1GXsnFsAbIrsFygjyTbDg4CRzrkuwIZKjnsAGO6c6w6cA/j/uT3MbEyG4/cBPgq9Xl/6nsQvqjIGGAUMBdQ3LFlRlnGskpzPcI1zblk1jusJHBhaO3d3M2vqnFsCLIksOsmGSMrYzPoDHznnVphZz+yFK7WQN9/jJCvDLaHtnUB4pfAmoW2jZo20G4AOodf7UvlfLIlOVGV8NDDAzPqWnqeVmU12zg2qU7RSG1GVcexyomtNaaPrZjPb38waAGeGds8FrvQvzKyqhtTFQBczKzCzH1GSks/KdsxSM9ksY+fcMOfcvs65QuAC4DlVhMnL8vc4djlRGZa6HpgDLKKknc+7EjjGzN4wsyLgUqi4rcE5tx24CngeKAKmOOfeiTp4qZaslLHktKyVsZlNAxZSktysN7NfRxm4xiaLiJBbmaGISGJUGYqIoMpQRARQZSgiAtSwn2GbNm1cYWFhRKHkng8++IDi4mKr+sj8oTLOfyrjzGpUGRYWFrJsWXU6m+eH7t27Jx1C7FTG+U9lnJluk0VEUGUoIgKoMhQRAVQZiogAqgxFRABVhiIigCpDEREg2cldRUQA2Lx5MwAffvhhhccUFBQAMHLkSAC6du0KwAEHHADAIYccUqcYlBmKiJBwZvjpp58CcM455wBw9NFHA3DZZZcBJT3ls+GLL74A4KWXXgLglFNOAaBRo0ZZOb+I1MxTTz0FwOzZswF48cUXAXjvvfcq/MyBBx4IlAyvA9i2bVva/p0767ZKqTJDERESyAx92wDAT37yEyDI3Pbaay8g+xnhYYcdBkBxcTFAalzm/vvvn5XrSPV9+eWXAPzpT38CYNWqVQDMnTs3dYwy9vywZs0aAB566CEAxo0bl9q3detWAGoy0/4770S7eocyQxERYswMfVbm2wcBPv/8cwCuvLJk0azRo0dn9Zp33XUXAGvXrgWCv0zKCOM3ZcoUAG666Sag/FNDnzEC7LHHHvEFJpFZv75kPahRo0bV6TwHHXQQEDw9jooyQxERYswMX3vtNSB4ahR2yy23ZO06b775Zmp7xIgRAJx5Zsnyreeee27WriPV47ODa6+9FgjuEMzS59ocMmRIavvBBx8EoHXr1nGEKLXgyxGCzO/YY48Fgt4ajRs3BmDXXXcFoEWLFqnPfP311wD84he/AIKsr0ePHgAceuihqWObNm0KQPPmzbP8W6RTZigigipDEREghttk37H6iSeeKLdv4sSJALRt27bO1/G3x7169Sq3b8CAAQC0bNmyzteRmvFNFf5hWUWmTp2a2n7mmWeA4GGLv4X2t12SnC1btgDp37PXX38dgJkzZ6Yde9RRRwGwfPlyIL3LnH+Atu+++wLQoEHyeVnyEYiI5IDIM8M//OEPQNC1wneABjj77LOzdp2XX34ZgI8//jj13sUXXwzABRdckLXrSNXWrVuX2p40aVLaPj+Y3newf/7558t93neW91nl+eefD0C7du2yH6xUy3fffQfAr371KyDIBgFuvPFGAHr27Jnxs5kGUey3335ZjrDulBmKiBBDZui7UPif++yzT2pfXdqA/HCeu+++GwiG/IS7bPg2SYnXihUrUtu+M/Xxxx8PwIIFCwD49ttvAXjkkUcA+Otf/5r6zOrVq4Egy+/Xrx8QtCWqy018fBcY/z3zEyuE2/mHDh0KQLNmzWKOLruUGYqIkMBEDX7qHoDevXsDsNtuuwEwePDgKj/vO237n4sXL07bn812SKmd8NRKPlP3na69Jk2aAPCb3/wGgMcffzy1zw/w94P4fcahp8nx80+I77nnHiCYYHXhwoWpY3yn6vpOmaGICDFkhldffTUA8+fPB2Djxo2pfb79yGcATz75ZJXn88eWHc7VsWNHIGjbkOT861//Kvfev//9bwD69++f8TN+WrVMjjzySCB9OJfEY9GiRWmv/TA53z8wnygzFBEhhszw8MMPB2DlypVA+pPGZ599FoDhw4cDsOeeewIwaNCgCs934YUXAnDwwQenve+XDPAZoiTnvPPOS237bP/VV18F4O233waCfw8zZswA0if99W3I/j0/9Zov+y5dukQWu6QLt+VC8ET/9ttvT73Xt29fIH1yhfpImaGICKoMRUQAsJqsQdC9e3dXWUN3HN5//30guB3u1q0bAM899xyQnUkfvO7du7Ns2TKr+sj8kY0y3rRpU2rbl5MfYlfRA7DwwH/fgf70008H4N133wWCVRPHjBlTp/jCVMaVKztoIpOGDRsCcPnllwPBnIQfffQRAJ06dQKCNY/C/Bo4flKHKB7MVLeMlRmKiJDwusm1cccddwDBXyr/8CWbGaHUTXi43LRp0wA466yzgPIZ4lVXXQXAvffem/qM75Dtp17zQ/XmzJkDBJ2yQQ/MovbHP/4RgPvuu6/CY3bs2AEEGb3/WRP+4emJJ54IpE/pFhdlhiIi1JPM0GcXAJMnTwagVatWgFZSy3V+WiffRcNPzOC7z/hM32eDYTfffDMAb731FhB00/GfgeDfg0TDD8Pzq1r66dS2b9+eOsavc+MzxNrwk0D773p4JTw/yW/UlBmKiFBPMkPf0TPstNNOA9Ini5Xc5TPEiiYAzcSviuZXNfSZ4QsvvJA6xj+51rRe0fBPio844gggeLIfNm/ePCDIFm+77TYAli5dWuPr+bbk//znPzX+bF0pMxQRoR5mhn7tVP+US/Kfb6+aNWsWkP6k0a+xnM21t6VmTj755LTXfsitzwwbNWoEBMtwAFx66aUAjBw5EgjakpOkzFBEBFWGIiJAjt8m+2FX4RXv/KpqenDyw+HX1B02bBiQvj6vb6wfOHAgAAcccEC8wUk5fgZ7v2qef7DiZx8CeO+994BgxvqywmslxUWZoYgI9SQzDA8S79OnT9oxX331FRDMfZeL67FKdvhJOe68887Ue/5B2g033AAE63P7bjkSv86dOwNBl6hHH3203DHh7lEAu+xSUhX5LnPh4ZlxUWYoIkKOZ4aZ+L8gPgPwj+b98B0Nz8p/F110UWp77NixAEyfPh0I2qLKzoQu8fFZ+ahRo4Dg7i3ckfqTTz4BoLCwEAjK1LcBJ0GZoYgI9TAzHD9+PAATJkwA4Le//S0QDOqX/Beerm3u3LlAsJ6vn1ggFzrx/tD5nh9+rfR//vOfqX2vvPIKEGSCfgqvJCkzFBEhxzPD0aNHA3Drrbem3jv++OMBGDx4MAC77747AI0bN445OskFvveAXzbAD9krKioCtJJeLvGrG5bdzhXKDEVEyPHM8LjjjgNg/vz5CUciuc5PHnvIIYcAsHr1akCZoVSfMkMREVQZiogAOX6bLFJdfk2ctWvXJhyJ1FfKDEVEUGUoIgKoMhQRAcD8alTVOtjsM2BddOHknALnXNuqD8sfKuP8pzLOrEaVoYhIvtJtsogIqgxFRICI+xma2R7AvNKX7YAdwGelr3/mnPsuwmvvArwGvO+c6x/VdX7okipjM7sOuKT05Rjn3OgoriOJlvF6YHPp9bY553pEcZ3U9eJqMzSz24CvnXMjyrxvpXHszPL1hgHdgGaqDOMRVxmbWTdgMnAk8D3wHPAb55x6XEcszu9xaWXY1Tn3v2ydszKJ3CabWSczKzKzh4FVQAcz+19o/0Azm1C6vZeZTTezZWa21MyOrMb5C4BewKSofgepXMRl3BlY7Jzb6pzbDrwEnBnV7yKZRf09jluSbYYHASOdc12ADZUc9wAw3DnXHTgH8P9ze5jZmAo+MwoYCuhRebKiKuOVwAlm1trMmgOnAh2yG7pUU5TfYwfMN7P/mNklFRyTNUmOTV7jnFtWjeN6AgeGlgvd3cyaOueWAEvKHmxm/YGPnHMrzKxn9sKVWoikjJ1zb5rZ/cBc4GtgOSXtShK/SMq41JHOuQ1m1g543szecs4tykLMGSVZGW4Jbe8ELPS6SWjbqFkj7dHAADPrW3qeVmY22Tk3qE7RSm1EVcY458YB4wDMbDiwug5xSu1FWcYbSn9+bGZPAj8DIqsMc6JrTWmj62Yz29/MGpDe/jMXuNK/KG08r+xcw5xz+zrnCoELgOdUESYvm2VcesyepT8Lgb7A1GzGKzWXzTI2sxZm1qJ0uzklzwDezH7UgZyoDEtdD8yhpOZfH3r/SuAYM3vDzIqAS6HKtgbJTdks45mlx84ELnfOfRlh3FJ92Srj9sD/mdnrwFJghnNubpSBazieiAi5lRmKiCRGlaGICKoMRUQAVYYiIoAqQxERQJWhiAigylBEBFBlKCICwP8D3P5bzM0W5d8AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7fcc687e6a20>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Get the first images from the test-set.\n",
    "images = data.x_test[0:9]\n",
    "\n",
    "# Get the true classes for those images.\n",
    "cls_true = data.y_test_cls[0:9]\n",
    "\n",
    "# Plot the images and labels using our helper-function above.\n",
    "plot_images(images=images, cls_true=cls_true)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Estimator 的输入函数"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "相较于直接给Estimator直接提供原始数据，我们必须提供一个函数来处理这些数据再输入。这允许在数据源上有更大的灵活性，例如可以对数据进行随机的清洗和迭代。\n",
    "\n",
    "注意到我们将利用`DNNClassifier`来创建一个Estimator，它默认类别数为整形，所以我们要用`data.train.cls` 来代替`data.train.labels`（独热数组）。\n",
    "\n",
    "这个函数为了能更好的控制数据的读取包含 `batch_size`, `queue_capacity` 和 `num_threads`参数。在这个例子中我们直接从内存中读取数据的numpy数组，所以在这里不需要这些参数。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "train_input_fn = tf.estimator.inputs.numpy_input_fn(\n",
    "    x={\"x\": np.array(data.x_train)},\n",
    "    y=np.array(data.y_train_cls),\n",
    "    num_epochs=None,\n",
    "    shuffle=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "这实际上返回一个函数："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<function tensorflow.python.estimator.inputs.numpy_io.numpy_input_fn.<locals>.input_fn>"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train_input_fn"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "调用这个函数返回一个TensorFlow的输入和输出数据的操作的元组："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "({'x': <tf.Tensor 'random_shuffle_queue_DequeueMany:1' shape=(128, 784) dtype=float64>},\n",
       " <tf.Tensor 'random_shuffle_queue_DequeueMany:2' shape=(128,) dtype=int64>)"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train_input_fn()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "相同的，我们也需要需要创建一个用于读取测试集的函数。注意到我们只想要处理这些图像1次所以`num_epochs=1`并且我们不希望图像的顺序被打乱所以 `shuffle=False`。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "test_input_fn = tf.estimator.inputs.numpy_input_fn(\n",
    "    x={\"x\": np.array(data.x_test)},\n",
    "    y=np.array(data.y_test_cls),\n",
    "    num_epochs=1,\n",
    "    shuffle=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "还需要一个输入函数来预测新数据的类。作为一个例子我们只取了了测试集中的一部分图像。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "some_images = data.x_test[0:9]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "predict_input_fn = tf.estimator.inputs.numpy_input_fn(\n",
    "    x={\"x\": some_images},\n",
    "    num_epochs=1,\n",
    "    shuffle=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "在这个输入函数中正确的类别数实际上没有被用到，因为在预测阶段这是不需要的。然而，正确的类别数在我们之后画图中需要用到。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "some_images_cls = data.y_test_cls[0:9]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##  预制/封装 Estimator\n",
    "\n",
    "当我们使用预制Estimator时，我们需要为数据指定输入特性。在这个例子中，我们希望从我们的data-set中输入图像，它们是给定形状的数字数组。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "feature_x = tf.feature_column.numeric_column(\"x\", shape=img_shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "你可以有几个输入特性，然后将它们组合在一个列表中："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "feature_columns = [feature_x]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "在这个例子中我们想要用3个隐藏层的DNN，分别有512.256，128单元。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "num_hidden_units = [512, 256, 128]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "用`DNNClassifier`构筑我们的神经网络。我们也可以指定激活函数和其他不同的参数（见文档）。这里我们只指定分类的类别数量和checkpoints保存的地址。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Using default config.\n",
      "INFO:tensorflow:Using config: {'_model_dir': './checkpoints_tutorial17-1/', '_tf_random_seed': None, '_save_summary_steps': 100, '_save_checkpoints_steps': None, '_save_checkpoints_secs': 600, '_session_config': None, '_keep_checkpoint_max': 5, '_keep_checkpoint_every_n_hours': 10000, '_log_step_count_steps': 100, '_train_distribute': None, '_device_fn': None, '_service': None, '_cluster_spec': <tensorflow.python.training.server_lib.ClusterSpec object at 0x7fcc66415240>, '_task_type': 'worker', '_task_id': 0, '_global_id_in_cluster': 0, '_master': '', '_evaluation_master': '', '_is_chief': True, '_num_ps_replicas': 0, '_num_worker_replicas': 1}\n"
     ]
    }
   ],
   "source": [
    "model = tf.estimator.DNNClassifier(feature_columns=feature_columns,\n",
    "                                   hidden_units=num_hidden_units,\n",
    "                                   activation_fn=tf.nn.relu,\n",
    "                                   n_classes=num_classes,\n",
    "                                   model_dir=\"./checkpoints_tutorial17-1/\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 训练\n",
    "\n",
    "我们现在可以给定的迭代次数训练模型。这将自动加载并保存checkpoints，这样我们就可以在以后继续训练。\n",
    "\n",
    "注意到`INFO:tensorflow:`在每一行都被打印，这有阻碍快速阅读进展。它应该被单独的打印一行。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Calling model_fn.\n",
      "INFO:tensorflow:Done calling model_fn.\n",
      "INFO:tensorflow:Create CheckpointSaverHook.\n",
      "INFO:tensorflow:Graph was finalized.\n",
      "INFO:tensorflow:Running local_init_op.\n",
      "INFO:tensorflow:Done running local_init_op.\n",
      "INFO:tensorflow:Saving checkpoints for 0 into ./checkpoints_tutorial17-1/model.ckpt.\n",
      "INFO:tensorflow:loss = 300.61185, step = 0\n",
      "INFO:tensorflow:global_step/sec: 453.729\n",
      "INFO:tensorflow:loss = 33.910957, step = 100 (0.221 sec)\n",
      "INFO:tensorflow:global_step/sec: 545.745\n",
      "INFO:tensorflow:loss = 38.821697, step = 200 (0.183 sec)\n",
      "INFO:tensorflow:global_step/sec: 510.96\n",
      "INFO:tensorflow:loss = 36.428062, step = 300 (0.196 sec)\n",
      "INFO:tensorflow:global_step/sec: 509.188\n",
      "INFO:tensorflow:loss = 10.77646, step = 400 (0.196 sec)\n",
      "INFO:tensorflow:global_step/sec: 525.229\n",
      "INFO:tensorflow:loss = 20.211845, step = 500 (0.190 sec)\n",
      "INFO:tensorflow:global_step/sec: 529.656\n",
      "INFO:tensorflow:loss = 16.973766, step = 600 (0.189 sec)\n",
      "INFO:tensorflow:global_step/sec: 518.829\n",
      "INFO:tensorflow:loss = 9.104766, step = 700 (0.193 sec)\n",
      "INFO:tensorflow:global_step/sec: 517.877\n",
      "INFO:tensorflow:loss = 11.87432, step = 800 (0.194 sec)\n",
      "INFO:tensorflow:global_step/sec: 513.369\n",
      "INFO:tensorflow:loss = 7.3187075, step = 900 (0.194 sec)\n",
      "INFO:tensorflow:global_step/sec: 531.02\n",
      "INFO:tensorflow:loss = 5.238852, step = 1000 (0.188 sec)\n",
      "INFO:tensorflow:global_step/sec: 493.925\n",
      "INFO:tensorflow:loss = 6.4892335, step = 1100 (0.203 sec)\n",
      "INFO:tensorflow:global_step/sec: 513.837\n",
      "INFO:tensorflow:loss = 10.295633, step = 1200 (0.194 sec)\n",
      "INFO:tensorflow:global_step/sec: 516.007\n",
      "INFO:tensorflow:loss = 4.5178833, step = 1300 (0.194 sec)\n",
      "INFO:tensorflow:global_step/sec: 501.485\n",
      "INFO:tensorflow:loss = 2.4612594, step = 1400 (0.200 sec)\n",
      "INFO:tensorflow:global_step/sec: 508.118\n",
      "INFO:tensorflow:loss = 10.878417, step = 1500 (0.197 sec)\n",
      "INFO:tensorflow:global_step/sec: 505.549\n",
      "INFO:tensorflow:loss = 22.480297, step = 1600 (0.198 sec)\n",
      "INFO:tensorflow:global_step/sec: 512.93\n",
      "INFO:tensorflow:loss = 6.8385906, step = 1700 (0.195 sec)\n",
      "INFO:tensorflow:global_step/sec: 520.968\n",
      "INFO:tensorflow:loss = 1.8562572, step = 1800 (0.192 sec)\n",
      "INFO:tensorflow:global_step/sec: 547.812\n",
      "INFO:tensorflow:loss = 4.875979, step = 1900 (0.183 sec)\n",
      "INFO:tensorflow:Saving checkpoints for 2000 into ./checkpoints_tutorial17-1/model.ckpt.\n",
      "INFO:tensorflow:Loss for final step: 2.701511.\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<tensorflow.python.estimator.canned.dnn.DNNClassifier at 0x7fcc663a5160>"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.train(input_fn=train_input_fn, steps=2000)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 评估\n",
    "\n",
    "一旦训练好模型，我们可以在测试集上评估它的性能。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Calling model_fn.\n",
      "INFO:tensorflow:Done calling model_fn.\n",
      "INFO:tensorflow:Starting evaluation at 2018-07-16-11:23:09\n",
      "INFO:tensorflow:Graph was finalized.\n",
      "INFO:tensorflow:Restoring parameters from ./checkpoints_tutorial17-1/model.ckpt-2000\n",
      "INFO:tensorflow:Running local_init_op.\n",
      "INFO:tensorflow:Done running local_init_op.\n",
      "INFO:tensorflow:Finished evaluation at 2018-07-16-11:23:09\n",
      "INFO:tensorflow:Saving dict for global step 2000: accuracy = 0.972, average_loss = 0.09360652, global_step = 2000, loss = 11.848927\n",
      "INFO:tensorflow:Saving 'checkpoint_path' summary for global step 2000: ./checkpoints_tutorial17-1/model.ckpt-2000\n"
     ]
    }
   ],
   "source": [
    "result = model.evaluate(input_fn=test_input_fn)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'accuracy': 0.972,\n",
       " 'average_loss': 0.09360652,\n",
       " 'global_step': 2000,\n",
       " 'loss': 11.848927}"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "result"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Classification accuracy: 97.20%\n"
     ]
    }
   ],
   "source": [
    "print(\"Classification accuracy: {0:.2%}\".format(result[\"accuracy\"]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 预测\n",
    "\n",
    "训练好的模型也可以拿来预测新的数据。\n",
    "\n",
    "注意到在对新数据预测时，TensorFlow图被再次创建并且加载checkpoint一次。如果模型非常大，这开销会是非常大的。\n",
    "\n",
    "不知道为什么Estimator被设计成这种方式，可能是为了总是能加载最近的一个Checkpoint，它也可以很容易地分布在多台计算机上。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [],
   "source": [
    "predictions = model.predict(input_fn=predict_input_fn)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Calling model_fn.\n",
      "INFO:tensorflow:Done calling model_fn.\n",
      "INFO:tensorflow:Graph was finalized.\n",
      "INFO:tensorflow:Restoring parameters from ./checkpoints_tutorial17-1/model.ckpt-2000\n",
      "INFO:tensorflow:Running local_init_op.\n",
      "INFO:tensorflow:Done running local_init_op.\n"
     ]
    }
   ],
   "source": [
    "cls = [p['classes'] for p in predictions]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([7, 2, 1, 0, 4, 1, 4, 9, 6])"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cls_pred = np.array(cls, dtype='int').squeeze()\n",
    "cls_pred"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAU0AAAD5CAYAAACj3GcTAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xe4VdW19/HvQDAIKIjYKScRC0gUE+yxXAtRVFBiFGPhGtEoxhJvIIlRggkag43XEkV9RG+wgopIRBS70oSASrGAokEvIqJGUbGN94+95t7r9L3O7off53nOc3ZZZcA8e+4x55prTnN3REQkOy1KHYCISCVRpSkikoAqTRGRBFRpiogkoEpTRCQBVZoiIgmo0hQRSUCVpohIAqo0RUQSaJnLzp06dfKqqqo8hVIZ5s2bt9rdNy91HMWiMm7+VMbJ5FRpVlVVMXfu3FwOUXHM7O1Sx1BMKuPmT2WcjJrnIiIJqNIUEUlAlaaISAKqNEVEElClKSKSQE5Xz0Wa6sorrwTgiy++AODll18GYOLEibW2PeusswDYe++9ATj55JOLEaJInZRpiogkoExTiur4448HYMKECXW+b2a1XrvpppsAmD59OgAHHHAAAF27di1EiFJCr7/+OgA77rgjANdeey0A55xzTsliqkmZpohIAso0peBCdgn1Z5g77bQTAIcddhgAb775Zvq9yZMnA7B06VIAxo8fD8CFF16Y/2ClpObPnw9AixapfG7bbbctZTh1UqYpIpKAMk0pmHA/84MPPljrvV69egGZLLJTp04AtGvXDoCvvvoqve2ee+4JwEsvvQTAhx9+WKCIpdQWLFgAZP4OBg4cWMpw6qRMU0QkgaJnmmEc3i233ALANttsk36vdevWAJx44okAbLXVVgB07969mCFKnvzf//0fAO6efi1kmNOmTQNg6623rnPfMI4TYMmSJdXeO/LII/Map5TeK6+8AsB1110HwCmnnFLKcBqkTFNEJIGiZ5rDhg0DYPny5fVuE8blbbLJJgD07NkzL+fu0qULAMOHDwegT58+eTmu1O2oo44CMle9ATbeeGMAOnbs2OC+9957b/pxvH9TmqfXXnsNgLVr1wLVR1yUG2WaIiIJqNIUEUmg6M3zW2+9FcgMH4k3vRcvXgxkBrg+/fTTAMyaNQvI3Db3zjvv1Hv8Vq1aAZkhLOFiRPw4oZmu5nlxdOvWLettr7jiCiBzO11cGHoUfkvzMXr0aCC19AaU92dTmaaISAJFzzQPPvjgar/jwi10wUcffQRkMs/w7fPiiy/We/zvfe97QOaG/3B7HsCaNWsA2G677ZoUuxTOlClTABgxYgQA69atS7+35ZZbAnD55ZcD0KZNmyJHJ4UQvxgcPtPhc9u2bdtShJQVZZoiIgmU9W2Um266KQAHHXRQtdfrylJruv/++4FMtgqwyy67ADBo0KB8hSh5Em65jGeYQRh+EqaEk+bhmWeeqfXa5puX/3LzyjRFRBIo60yzKVatWgXA0KFDgeq38IX+ssYGVkvxHH300UDmtspg8ODB6cejRo0qakxSHGGJk7hw40k5U6YpIpJAs8s0b7jhBiCTcXbo0CH9XrgyJ6UXxs/OmDEDyPRlhj6tiy66KL1tmCZMmoeZM2cCMG7cuPRru+22GwCHHnpoSWJKQpmmiEgCzSbTfP7554HMWL7goYceSj8O05JJ6YXJZVevXl3t9TAtoMbSNl9PPPEEUH1kSxijHaaHLGfKNEVEElClKSKSQLNpnj/yyCNAZu7FQw45BIC99967ZDFJbWFNoHBrbHDggQcC8Oc//7nYIUmRhcl64n7+85+XIJKmUaYpIpJAxWeaX3zxBQCPPvookJmw45JLLgEyU8VJ6cRXj7zsssuA2rOx9+7dG9DwouZs5cqVADz33HNA9cl0jjnmmJLE1BTKNEVEEqj4TDNMWhv6yA4//HAA9tlnn5LFJNVdddVV6cdz5syp9l64jVJ9mc3f7bffDsD7778PZD6rlUaZpohIAhWZaYYJawH+8pe/ANC+fXsALr744pLEJPW7+uqr630v3Paqvszm7+233672PEz9WGmUaYqIJFBRmWa4CnvuueemX/vmm28A6NevH6BxmZUmlGk2oxxCayJs+/XXXwPwySef1No23KJ3zTXX1HmsDTbYIP34b3/7G6BlNArt4Ycfrvb8yCOPLFEkuVGmKSKSgCpNEZEEKqJ5/u233wKZmVDeeuut9Hvdu3cHMheEpLKEdZuycdxxxwGw9dZbA5mhK/fcc09OMYTVLuNzeEr+hMHsobwqnTJNEZEEKiLTXLZsGZBZsTAuDGfR/IvlK1ykA5g0aVKTj3Pfffc1uk24SNSiRfV8oH///gD06dOn1j4/+clPmhyTNO7BBx8EMhdtwyztlbq6qDJNEZEEyjrTDINh+/btW+31K6+8Mv24UoctrE8eeOCB9OPRo0cDtSfsCBYvXgw03E952mmnAdCtW7da7/3sZz8DoEePHk0LVvLm888/B2Dq1KnVXg/TwMWHfVUSZZoiIgmUdaY5duxYoPbtV/G+EDMrakySm2zXtb7rrrsKHIkUWuhfDivCDhgwAIDzzjuvZDHlgzJNEZEEyjLTDOO6rr/++hJHIiJNFTLNsM55c6FMU0QkgbLMNMMa5p9++mm118PdP5pGTERKRZmmiEgCqjRFRBIoy+Z5TWGlwieeeAKAjh07ljIcEVmPKdMUEUmgLDPNP/zhD9V+i4iUC2WaIiIJmLs3fWezD4C3G92weenm7puXOohiURk3fyrjZHKqNEVE1jdqnouIJKBKU0QkgQYrTTPbzMwWRD8rzezd2PMNCxGQmfWMnWOBmX1qZr9uZJ8hZvZBtP0SM/tljjGMN7OjG9nm97EYF5nZN2bWPpfzlkKJyribmT1tZouj/7sGyzfapxRlfIqZvWJmL5vZC2b2w1zOWSqlKOPovHeEMsty+1KU8c5mNtPM1pnZ+Vkd2N2z+gFGAr+t43UDWmR7nCQ/QCtgFdC5ke2GAGOix1sBq4FONbZpmeC844GjE2x/DPBYIf4PivlTrDIGtgF6R483AZYBO5RbGQP7Ah2ix0cBL5S6jCqljKNjHgDsASzIcvtSlPGWQB/gcuD8bI7bpOa5mXWPsoQ7gUVAFzP7OPb+IDO7NXq8pZk9YGZzzWyOme2V4FSHAkvcfUW2O7j7SmA50NXMRpnZ/5rZC8DtZtbSzK6O4njZzIZEMbYws7+b2atm9jjQKUGMACcAdyfcp6wVsozd/T13XxA9/g/wKrBttrEVq4zd/QV3D//mWUDnbGOsBIX+HLv7M8CapsRWxDJ+393nAt9kG1sug9t3Ak5x97lm1tBxrgVGu/ssM6sCpgC9zGxP4FR3P7OBfQeRsDIys+5AN+DNWJz7u/uXZjYUWOXue5jZ94BZZvYYsBfwfaAnqSxoMXBTdLxLSWUYj9RzvnbAIcDpSeKsEAUvYzP7AdALeDHboIpdxpHTgKkNvF+pivE5TqxEZZyVXCrNZVEN3ZhDgB0tsyzFpma2kbvPBmbXt5OZtQaOAC7IMp4TzexAYB0wxN0/js75kLt/GW3TF+hhZoOi5+2B7YH9gbvd/TtghZk9HQ7q7n9s5LwDgGfc/ZMs46wkhS7jTYD7gXPc/bMszlOSMjazQ4CTgea41m9By7gJSvU5zlouleba2OPvSPWJBK1jjw3Yw93rXn6wfkcAs919dZbb3+nudXXkxuM0YKi7PxHfwMyOSRhb3CDgHznsX84KVsaWugDxADDO3SdnuVvRy9jMegNjgZ+6+0dNOUaZK/TnOKlSfY6zlpchR1HN/pGZbW9mLUhdGAmmA2eHJ9EfYTZq9ROa2XlmlkszYBowNDRDzGxHM9sIeBY4PuoT2ZZUB3ajzGxTYB/g4Rxiqgj5LGNLpQ63k7pAcG2N98qmjKNm6ETgF+6+NIeYKkKBPse1lFMZN0U+x2n+jtQ/ZgYQv3BzNrBv1GG7mKjvz8z2NLOb6jqQmW0M/BcwqcZbPYAPc4hxLPAGsMDMFgI3ksq2JwLvkOoDGQekFzUxs0vNrF89x/sZMNXdv8ghpkqSrzI+gNSX4qGWGfry0+i9cirjkUBHYGwUYz6boeUqn5/jCcBzQE8zW2Fm/x29VTZlbGadzWwFcC4wMoqzTUMnr6jbKM3sn8AAd8/6SpdUFpVx81fpZVxRlaaISKnpNkoRkQRUaYqIJKBKU0QkAVWaIiIJ5LRGUKdOnbyqqipPoVSGefPmrfb1aFZvlXHzpzJOJqdKs6qqirlzs7kDq/kws/VqWQCVcfOnMk5GzXMRkQRUaYqIJKBKU0QkAVWaIiIJqNIUEUlAlaaISAI5DTkqlrVrU/OPDhs2DICbbsrMRNWnTx8AJkyYAEC3bt2KHJ2IrE+UaYqIJFARmeZ7770HwC233ALABhtskH4vDMp9+OHU5Om//nWjS2hLGfjXv/4FwMCBAwFYvnx5k4/12GOPpR/36NEDgC5dujQ9OCmZ8Dnu378/ANdddx0AZ511Vnqb+Oe/FJRpiogkUNaZ5gcffADA4MGDSxyJ5Nu0adMAWLduXc7Hmjw5sy7bbbfdBsA999yT83GleD78MLX6RTyjBDjnnHMAOO2009KvbbTRRsULrA7KNEVEEijLTPPaa1MLFE6alFpX7cUXX2x0n+eeew6AsHzHrrvuCsD+++9fiBClib75JrUszCOPPJK3Y4YRFABXX301kBlx0bZt27ydRwrn2WefBeDdd9+t9voJJ5wAQOvWrWvtUyrKNEVEEijLTPP881NrxSe5SvbAAw9U+921a1cA7rvvvvQ2P/7xj/MVojTRU089BcCMGTMA+N3vfpfzMdesWZN+vGjRIgA+//xzQJlmOYv3Z48aNarObU4++WQAzKwoMWVDmaaISAKqNEVEEiir5nm/fv2AzMWcb7/9ttF9OnXqBGSaYW+/nZqQ+a233gJg9913T2/73Xff5S9Yydorr7ySfjxo0CAAunfvDsCFF16Y8/HjQ46kcrz88svpx+Fmh6Bly1TVdPjhhxc1pmwo0xQRSaDkmeYzzzyTfvzqq68CmU7f+i4EnXnmmenHffv2BaB9+/YAPPnkkwBceumltfa78cYbgdoDaKWw4mURLtCMHz8egHbt2jX5uOECUPxvqJwuGEjDwkXbuhx66KFFjCQZZZoiIgmULNMMEzSEPi6A1atX17ltGD507LHHAvCnP/0p/V6bNm2qbRumhhs7dmytYw4fPhyAL7/8EshM7tGqVaum/SOkQRMnTgSqD2QPfZnxvuamCsNU4tnlgQceCECHDh1yPr4UVryFEGy44YYAXHbZZcUOJ2vKNEVEEihZpvn1118D9WeXkLkF8t577wUyV8obEjLNcFX2ggsuSL8Xbq0LGWeYfmq77bZLFLtkJ0wMHf7fIT/9yaGVctdddwGZK60AF110EaDWQzkLNzbMnDmz1nuh5di7d++ixpSEMk0RkQRKfvW8LqG/a9y4cUB2GWZNIYu8884706/NmTMnD9FJYz755BMAZs2aVeu9oUOH5nz8m2++GchMHdizZ8/0ewcddFDOx5fCamgCnkoY2aJMU0QkgZJnmnXd9TN79uycjxvuKorfBVTzTqNwFT6MGZT8CBMxrFixAshM75Uvy5Ytq/a8V69eeT2+FFZdmWYY7ZCPlkihKdMUEUlAlaaISAIla56HtcsLtbJcWNVu/vz56ddq3p55ySWXFOTc67uNN94YyAwbiU/YEW597NixY+Ljrlq1CsgMZQr23XffJsUpxfX8888DmaFiceE26M6dOxc1pqZQpikikkDJMs0pU6bk9Xhh+MnixYuBhm/DCkOYNAC6MMJqgeGWyXA7JcARRxwBVL/poC4LFy5MPw4XfsK0fzUn5WjRQt/9lSCsOBkuyMaV8wQdNemvTUQkgZIPOcqXMP3YDTfcUO82VVVVANxxxx1AZiIQKYyRI0cC1TOL0MKIT9RSl8033zz9OGSW9d1ye+qpp+YSphRJzb7o+KQqZ5xxRrHDaTJlmiIiCVR8phmWyAgTGDck3G633377FTQmSenRowdQfUXQMJqh5gD1msI0gHGDBw8Gat+MEPpQpTyFmxxqXjWPXynPx1SBxaJMU0QkgZJlmg0tnjZ16tRqz08//XQA3nvvvXqPk80yB/m+Yi/J7bbbbtV+J/GDH/ygztfj40B/+MMfNi0wKZgwFVzNq+YDBgwoRTg5U6YpIpKAKk0RkQRK1jwP8+aFWdTjwgDomrdY1nXLZWjeZ7NypVS20Lyr2cxTk7y8hUHtQbi55Pzzzy9FODlTpikikkDJMs2BAwcCMHr06PRrDa0X1Jjw7RWGudxyyy0AbL311k0+ppSXcLFPa5tXlmnTplV73qVLFyAzSUelUaYpIpJAyTLNsGpkWGkSYNKkSQCMGTMm8fH++Mc/Apm1zKX5CevVBxrUXt7CirNLly6t9nrr1q2Byp0wR5mmiEgCJb+NMqxtHn/ct29fILPqYJhQ+KijjgLgV7/6VXqfcCU1viKhNE9hddIw0cOIESNKGY40IkzZF26RXLRoEQDbb799yWLKB2WaIiIJlDzTrMthhx1W7bcIZDKW3/zmN4DWOC93Yex0mLYxjHr40Y9+VLKY8kGZpohIAmWZaYrUJfRtS2XZZpttALjttttKHEl+KNMUEUlAlaaISAKqNEVEElClKSKSgCpNEZEEVGmKiCRgNSd0TbSz2QfA2/kLpyJ0c/fNG9+seVAZN38q42RyqjRFRNY3ap6LiCSgSlNEJAFVmiIiCTRYaZrZZma2IPpZaWbvxp5vWKigzKyfmb1mZkvNbFgW24+KxfaKmR2R4/mfN7PejWzT2swmRjHONLOuuZyzVEpVxtG5W5rZy2Y2KYtti17GsW2PNzPPdvtyU8LP8R1m9oGZLchy+yFhezNbYma/zPH8483s6Ea22Tn6/K4zs6yWx2xwwg53/xDoHR18JPCZu19Z46RG6oLSd9mcsDFm1gq4HvgvYCUw18wecvfXG9n1CncfY2a9gKfMbAuPXeUys5bu/k0+YoycAax09+5mdhLwV+DEPB6/KEpRxjEXAAuBNlluX+wyxsw2AYYCc/N53GIqYRnfBtwA3Jxgnzvd/Xwz2wpYaGaT3T294mIByng1cA5wbLY7NKl5bmbdzWyxmd0JLAK6mNnHsfcHmdmt0eMtzewBM5trZnPMbK9GDr8XsMTd33b3dcB9wIBsY3P3hYABm0bfNDea2RzgMjNrZ2a3R3HMN7OjohjbmNmE6NvtfqB1FqcaANwRPb4P+Gm2MVaCApcxZtYNOBQYlzS2IpYxwGXRz7qkcZa7Qpexuz8DrGlKbO6+ElgOdI1aGf9rZi8At0ctlKujOF42syFRjC3M7O9m9qqZPQ50yuI877v7XCDrijiXPs2dgGvcvSfwbgPbXQuMdvc+wHFAKIQ9zeymOrbfFvh37PmK6LWsmNk+wJfuHgpra2Avdx8OjAAedfc9gIOAq8ysNfBr4CN37wGMAnaLHW9cPc2ydJzu/hWw1sw6ZBtnhShUGQOMAYYBice8FauMzWx3YAt3n1bzvWakkGXcZGbWHegGvBmL82B3P4lUK29VVMa7A2dbqnvsWOD7QE/gVGCf2PEuNbN++Ygtl/k0l0U1dGMOAXa0zFrVm5rZRu4+G5idw/lrGmZm/w18Chwfe31CrMnRFzjczH4fPW8NdAX2B0YDuPt8M1sUdnb3U/MYY6UpSBlH/Uz/dvcFZnZIgniKVsZm1gK4igrsckmo3D7HJ5rZgaQy+yHu/nF0zofcPSxH2hfoYWaDouftge1JlfHd0d/CCjN7OhzU3f+YrwBzqTTXxh5/R6q5FMSbPgbsEWVj2XgX6BJ73pmGvwGDK9y9rrV/43EacLS7L4tvEPtDSCLEudJSnelt3f3jRvapNIUq432AgWbWPzrOJmZ2h7sPbmS/YpZxB1IZy3PRvlsBj5jZEe4+P+nBylihyrip7nT3ui7I1Czjoe7+RHwDMzumoJFF8jLkKKrZPzKz7aNv6Hjw04Gzw5N6mrpxs4CeZtbNzL5HqikwOdp3dOijaqJppDp9QyyhifYs8IvotV2BnbM41mQgfMiPAx7LIa6yl88ydvfh7t7Z3auAk4DHQoVZLmXs7mvcvZO7V0VxzgX6NbMKs5o8f47rZWbnmdmZTY+UacBQM2sZHW9HM9uIVBkfH/VtbgsckMM56pXPcZq/I/WPmUGqHzI4G9g36rBdDJwO9feFuPvXwLnA48BiYLy7vxa9vQupK+pNdQnQ1lJDVhYBI6PXrwc2M7MlwMVA+oPRQJ/mzcDWZraUVH/ZhTnEVSnyUsaNKKcyXh/lrYzNbALwHKkkaEXUtQLQA/gwhxjHAm8AC8xsIXAjqVbzROAdUvXGOGBmLJY6+zTNrLOZrSBV54yM4mxwNEfF3HtuqTbSVHfXEpXNlMp4/WBm/wQG5Ht4WLFUTKUpIlIOdBuliEgCqjRFRBJQpSkikkAu4zTp1KmTV1VV5SmUyjBv3rzV69Os3irj5k9lnExOlWZVVRVz51bsPAZNYmbr1bIAKuPmT2WcjJrnIiIJqNIUEUlAlaaISAKqNEVEElClKSKSgCpNEZEEVGmKiCSQ0zhNEZFC+OijjwB455136t2mW7duAFxzzTUA9OrVC4AddtgBgF133bUgsSnTFBFJoKwyzVWrVgFw3HHHAbDPPql1kc444wwgdedCPnzyyScAPPvsswAcdlhq+sZWrVrl5fgiksyUKVMAePjhhwF4+umnAXjjjTfq3WfHHXcEYPny5QCsW1d9wdDvvsv3itMpyjRFRBIoeaYZ+i4Adt45tWxLyAS33HJLIP8Z5o9+9CMAVq9OrUEf7rvdfvvt83Ieyd5//vMfAH7/+9TikYsWpRaJnD59enobtQCah2XLUmvd3XDDDQDcfPPN6fe++OILAJJMiv7aa681vlEBKNMUEUmgZJlmyPJC/yXAhx+m1lo6++zUonfXXXddXs85atQoAN566y0g802nDLP4xo8fD8BFF10E1L5KGjJQgM0226x4gUnBrFiRWqdtzJi6VmHO3k477QRkrpYXmzJNEZEESpZp/utf/wIyV8niRowYkbfzLFy4MP34yiuvBOCYY1LLOR9//PF5O49kJ2Qbv/nNb4BMiyO1EGXGOeekly7n+uuvB6Bjx47FCFGaIJQjZDLJn/zkJ0BmdMqGG24IQPv27QFo165dep/PPvsMgJ/+9KdAJovcc889Adhtt93S22600UYAtG3bNs//iuwo0xQRSUCVpohIAkVvnocB7Pfff3+t92677TYANt889+VZQrP80EMPrfXewIEDAdh4441zPo8kE7pIwkW/+txzzz3px1OnTgUyF41C0z0096R01q5dC1T/nL300ksATJo0qdq2e++9NwDz588Hqg8lDBcCO3fuDECLFuWbz5VvZCIiZajomeb//M//AJkhJ2GgOcDPf/7zvJ3n+eefB2DlypXp10499VQATjrppLydRxr39tuZNazGjRtX7b0wqUK4keHxxx+vtX+4KSFkqSeeeCIAW221Vf6Dlax89dVXAPziF78AMtklwIUXXgjAIYccUue+dd2s0rVr1zxHWDjKNEVEEih6phmGloTf2267bfq9XPqowm1Yl112GZC5VSs+lCX0mUpxLViwIP04DFrff//9AXjmmWcA+PLLLwG46667APjrX/+a3mfp0qVAptUwYMAAINPXqaFIxROGBoXPWZhgI34dYtiwYQC0adOmyNEVhzJNEZEESj5hR5gSCqBv374AdOjQAYCzzjqr0f3D4Pjwe9asWdXez2c/qTRNfMqukPmHwe1B69atAfjlL38JwMSJE9PvhYkewmQOIYPR1fPiC1fEL7/8ciAzEfBzzz2X3iYMXm+ulGmKiCRQ9EzzvPPOA+DJJ58E4L333ku/F/q3Qkbx0EMPNXq8sG3N2/C22247INP3IqVz991313rtn//8JwBHH310nfuE6frqstdeewHVb8OT4pgxY0a15+H2xjC+cn2gTFNEJIGiZ5o//vGPAXjllVeA6ldWH330UQBGjx4NwBZbbAHA4MGD6z3eySefDMAuu+xS7fWwVEbIOKV0TjjhhPTj0Hp48cUXAXj11VeBzN/Dgw8+CFSfnDr0cYfXwpR+oex79uxZsNilunhfM2RGMFxyySXp1/r37w9Un2SjOVGmKSKSgCpNEZEELMmaHDX16dPHG+qwL4Y333wTyDTDe/fuDcBjjz0G5Gfyjzgzm+fuffJ60DKWjzJes2ZN+nEop3BrZH0X8uITQIQbFY488kgAXn/9dSCzSulNN92UU3w1qYzrV/PmlLpssMEGAJx55plAZk7Mf//73wB0794dyKwJFhfWiAqTexTqAlMuZaxMU0QkgZIPbs/Vn//8ZyDzzRcuIuU7w5Smi9/mOGHCBACOPfZYoHbGee655wLwt7/9Lb1PGPgepvQLt1hOmzYNyAx+B134K7Tf/va3AFx11VX1bvPtt98CmRZC+J1EuAh84IEHAtWnCiw1ZZoiIglUZKYZshWAO+64A4BNNtkE0MqF5S5MFxaGroQJOsKwotByCNll3MUXXwzAkiVLgMzwpbAPZP4epDDC7ZNhFdkwTd/XX3+d3iasAxUyzqYIk5WHz3p85ckwGXWpKNMUEUmgIjPNMKA27ogjjgCqT2os5StknPVNVFuXsAphWEU0ZJpPPfVUeptwpV7TxRVGuDK+++67A5mRDHFPPPEEkMk+R44cCcCcOXMSny/0dc+bNy/xvoWiTFNEJIGKzzTD2sfhqp40f6E/bfLkyUD1K6thjfQRI0YUPzAB4OCDD672PNwqHTLNVq1aAZnlZwBOP/10AK655hog09ddjpRpiogkoEpTRCSBimqeh9vl4itMhlUMdQFo/RHWxB4+fDhQfX3tcNFh0KBBAOywww7FDU5qCSsyhFUqwwWiMFsVwBtvvAFkVmCoKb6WWKkp0xQRSaAiM834ZAH9+vWrts2nn34KZOZerKT1lCWZMDnLX/7yl/Rr4YLgH/7wBwDGjx8PZIYrSfH16NEDyAwVu/fee2ttEx82BtCyZapqCkMJ47cVUU8FAAAHAElEQVTVlpoyTRGRBCoq06xL+EYKGUUYshBuu9Jtdc3fKaeckn48duxYAB544AEg01dWc2Z/KZ6Q5Y8ZMwbItAbjA9bff/99AKqqqoBMmYY+6nKiTFNEJIGKzzRvueUWAG699VYAhgwZAmQmd5DmLz4N4PTp04HMetxhgolyHiy9vggjXaZMmQLAP/7xj/R7M2fOBDKZZZgarhwp0xQRSaCiMs3rrrsOgD/96U/p1/bff38AzjrrLAA23XRTADbccMMiRyflIIyWCMtlhFstFy9eDGjlynISVhOt+bjcKdMUEUmgojLN/fbbD4Ann3yyxJFIuQuTHO+6664ALF26FFCmKblTpikikoAqTRGRBCqqeS6SrbBm1FtvvVXiSKS5UaYpIpKAKk0RkQRUaYqIJGBhtbcm7Wz2AfB2/sKpCN3cffPGN2seVMbNn8o4mZwqTRGR9Y2a5yIiCajSFBFJoMFK08w2M7MF0c9KM3s39rygM2KYWUsze9nMJmWx7ahYbK+Y2RE5nvt5M+ud5bbHm5lnu325KVUZm9kFZrYo+jkni+2HmNkHUVxLzOyXOZ5/vJkd3cg2Hc1scvR3ONvMKvIezBKW8Yro87jAzGZnsX3Ryzja7mAzeyn6W2z0Hu0GB7e7+4dA7+jAI4HP3P3KGic0Un2j3zV2soQuABYCbbLc/gp3H2NmvYCnzGwLj3XYmllLd/8mnwGa2SbAUGBuPo9bTKUo4+gLZjDQB/gGeMzMprh7YyPR73T3881sK2ChmU1299Wx4+a7jC8GZrt7fzPbGfh/wKF5PH5RlPhzvJ+7f5xg+6KWsZl1BK4D+rr7CjNrdCLPJjXPzay7mS02szuBRUAXM/s49v4gM7s1erylmT1gZnPNbI6Z7ZXF8buR+uMclzQ2d18IGLBp9E1zo5nNAS4zs3ZmdnsUx3wzOyo6XxszmxB9u90PtM7ydJdFP+uSxlnuClzGPYBZ7v6Fu38NPAsck21s7r4SWA50jVoZ/2tmLwC3Ry2Uq6M4XjazIVGMLczs72b2qpk9DnTK4lQ9gSejcy4CdjCzzbKNs9wV+nOciyKW8UnAfe6+IjrvqsZ2yKVPcyfgGnfvCbzbwHbXAqPdvQ9wHBAKYU8zu6mefcYAw4DEl/bNbB/gS3dfE720NbCXuw8HRgCPuvsewEHAVWbWGvg18JG79wBGAbvFjjfO6mh6m9nuwBbuPi1pjBWkUGX8CnCApZq/bYHDgS7ZBmVm3YFuwJuxOA9295OAM4BVURnvDpxtZl2BY4Hvk6oITwX2iR3vUjOrvqxpykvAwGibvYHO0U9zUsjPsQNPmtk8MzstSVBFLOMdgM3M7JnoC+GkxmLL5d7zZe6eTbP0EGBHyyy7u6mZbeTus4Fa/RyW6oP4t7svMLNDEsQzzMz+G/gUOD72+oRYk6MvcLiZ/T563hroCuwPjAZw9/lmtijs7O6n1hFjC+Aq4MQE8VWigpSxuy80s6uB6cBnwHzg2yzOc6KZHUgqsx/i7h9H53zI3b+MtukL9DCzQdHz9sD2pMr47uhvYYWZPR2L54/1nO9S4FozW0CqAn0pyzgrSUHKOLKXu78bNbUfN7Ml7j6jkfMUu4xbAj8k1bJtC8w0s5nuvqy+AHOpNNfGHn9HqkkcxJu3Buzh7l9ledx9gIFm1j86ziZmdoe7D25kvyvcfUwjcRpwdM3/kNgfQrY6kPo2ey7adyvgETM7wt3nJz1YGStUGePuNwM3A5jZaGBpFrvd6e7nNxKnAUPd/Yn4BmaWdfM/FuMnpPpewxflcqC5zQBSyDJ+N/q90sweAvYAGqs0i1rGwArgXXf/HPg86gLYBai30szLkKOoZv/IzLaP/rjiwU8Hzg5P6mrq1jjWcHfv7O5VpPobHgsVppmNDv2QTTQNSF+pNbPQDH8W+EX02q7Azo3EuMbdO7l7VRTnXKBfM6swq8lnGUfbbBH9rgL6A/dEz88zszNzCHUaMNTMWkbH29HMNiJVxsdH/V7bAgdkEWMHM2sVPf0VMN3d1za0TyXLZxlb6vpBu+hxW1KZ3MLoedmUMTAJ2M/MNoji3AN4taEd8jlO83ek/jEzSNXewdnAvlGH7WLgdGi0L6Q+uwArc4jxEqCtpYZBLAJGRq9fT6pfYwmpK6bpyq++Ps31VD7LeFK07STgTHf/T/R6D+DDHGIcC7wBLDCzhcCNpFpUE4F3gMWkLjDODDs00N/1Q2Cxmb0GHExqREdzl68y3hp4wcxeAuYAD7r79Oi9sinj6MLxk6T62WcDf3f3JQ2dvGJuo7RUO3iqux9W6likcMzsn8CAfA8Pk/JR6WVcMZWmiEg50G2UIiIJqNIUEUlAlaaISAKqNEVEElClKSKSgCpNEZEEVGmKiCTw/wF5WJBSe+H5cQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7fcc65d22e48>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_images(images=some_images,\n",
    "            cls_true=some_images_cls,\n",
    "            cls_pred=cls_pred)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 新的Estimator"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "如果你不使用内置的Estimator，那么你可以自己创建一个任意TensorFlow 模型。为了完成这个，你首先需要创建如下的几个函数：\n",
    "1. TensorFlow 模型，例如一个卷积神经网络\n",
    "2. 模型的输出。\n",
    "3. 用于模型优化的损失函数。\n",
    "4. 优化的方法。\n",
    "5. 性能指标。\n",
    "\n",
    "这个Estimator可以运行三种模式：训练，评估，预测。这些代码大体相似，但是预测模式下我们不需要设置损失函数和优化器。\n",
    "\n",
    "这是Estimator API的另一个方面，它被设计的糟糕，就像我们过去用结构体来做ANSI C编程一样。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [],
   "source": [
    "def model_fn(features, labels, mode, params):\n",
    "    # Args:\n",
    "    #\n",
    "    # features: This is the x-arg from the input_fn.\n",
    "    # labels:   This is the y-arg from the input_fn,\n",
    "    #           see e.g. train_input_fn for these two.\n",
    "    # mode:     Either TRAIN, EVAL, or PREDICT\n",
    "    # params:   User-defined hyper-parameters, e.g. learning-rate.\n",
    "    \n",
    "    # Reference to the tensor named \"x\" in the input-function.\n",
    "    x = features[\"x\"]\n",
    "\n",
    "    # The convolutional layers expect 4-rank tensors\n",
    "    # but x is a 2-rank tensor, so reshape it.\n",
    "    net = tf.reshape(x, [-1, img_size, img_size, num_channels])    \n",
    "\n",
    "    # First convolutional layer.\n",
    "    net = tf.layers.conv2d(inputs=net, name='layer_conv1',\n",
    "                           filters=16, kernel_size=5,\n",
    "                           padding='same', activation=tf.nn.relu)\n",
    "    net = tf.layers.max_pooling2d(inputs=net, pool_size=2, strides=2)\n",
    "\n",
    "    # Second convolutional layer.\n",
    "    net = tf.layers.conv2d(inputs=net, name='layer_conv2',\n",
    "                           filters=36, kernel_size=5,\n",
    "                           padding='same', activation=tf.nn.relu)\n",
    "    net = tf.layers.max_pooling2d(inputs=net, pool_size=2, strides=2)    \n",
    "\n",
    "    # Flatten to a 2-rank tensor.\n",
    "    net = tf.contrib.layers.flatten(net)\n",
    "    # Eventually this should be replaced with:\n",
    "    # net = tf.layers.flatten(net)\n",
    "\n",
    "    # First fully-connected / dense layer.\n",
    "    # This uses the ReLU activation function.\n",
    "    net = tf.layers.dense(inputs=net, name='layer_fc1',\n",
    "                          units=128, activation=tf.nn.relu)    \n",
    "\n",
    "    # Second fully-connected / dense layer.\n",
    "    # This is the last layer so it does not use an activation function.\n",
    "    net = tf.layers.dense(inputs=net, name='layer_fc2',\n",
    "                          units=10)\n",
    "\n",
    "    # Logits output of the neural network.\n",
    "    logits = net\n",
    "\n",
    "    # Softmax output of the neural network.\n",
    "    y_pred = tf.nn.softmax(logits=logits)\n",
    "    \n",
    "    # Classification output of the neural network.\n",
    "    y_pred_cls = tf.argmax(y_pred, axis=1)\n",
    "\n",
    "    if mode == tf.estimator.ModeKeys.PREDICT:\n",
    "        # If the estimator is supposed to be in prediction-mode\n",
    "        # then use the predicted class-number that is output by\n",
    "        # the neural network. Optimization etc. is not needed.\n",
    "        spec = tf.estimator.EstimatorSpec(mode=mode,\n",
    "                                          predictions=y_pred_cls)\n",
    "    else:\n",
    "        # Otherwise the estimator is supposed to be in either\n",
    "        # training or evaluation-mode. Note that the loss-function\n",
    "        # is also required in Evaluation mode.\n",
    "        \n",
    "        # Define the loss-function to be optimized, by first\n",
    "        # calculating the cross-entropy between the output of\n",
    "        # the neural network and the true labels for the input data.\n",
    "        # This gives the cross-entropy for each image in the batch.\n",
    "        cross_entropy = tf.nn.sparse_softmax_cross_entropy_with_logits(labels=labels,\n",
    "                                                                       logits=logits)\n",
    "\n",
    "        # Reduce the cross-entropy batch-tensor to a single number\n",
    "        # which can be used in optimization of the neural network.\n",
    "        loss = tf.reduce_mean(cross_entropy)\n",
    "\n",
    "        # Define the optimizer for improving the neural network.\n",
    "        optimizer = tf.train.AdamOptimizer(learning_rate=params[\"learning_rate\"])\n",
    "\n",
    "        # Get the TensorFlow op for doing a single optimization step.\n",
    "        train_op = optimizer.minimize(\n",
    "            loss=loss, global_step=tf.train.get_global_step())\n",
    "\n",
    "        # Define the evaluation metrics,\n",
    "        # in this case the classification accuracy.\n",
    "        metrics = \\\n",
    "        {\n",
    "            \"accuracy\": tf.metrics.accuracy(labels, y_pred_cls)\n",
    "        }\n",
    "\n",
    "        # Wrap all of this in an EstimatorSpec.\n",
    "        spec = tf.estimator.EstimatorSpec(\n",
    "            mode=mode,\n",
    "            loss=loss,\n",
    "            train_op=train_op,\n",
    "            eval_metric_ops=metrics)\n",
    "        \n",
    "    return spec"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 创建Estimator的一个实例\n",
    "\n",
    "我们可以指定超参数，例如优化器的学习率。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [],
   "source": [
    "params = {\"learning_rate\": 1e-4}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我们可以创建这个新的Estimator的一个实例。\n",
    "\n",
    "注意我们没有提供feature-columns，因为它会当`model_fn()`被调用时，从data-functions自动推断出来。\n",
    "\n",
    "在TensorFlow文档中写的不清楚，为什么在上面使用`DNNClassifier`必须指定feature-columns，而这里不需要。 "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Using default config.\n",
      "INFO:tensorflow:Using config: {'_model_dir': './checkpoints_tutorial17-2/', '_tf_random_seed': None, '_save_summary_steps': 100, '_save_checkpoints_steps': None, '_save_checkpoints_secs': 600, '_session_config': None, '_keep_checkpoint_max': 5, '_keep_checkpoint_every_n_hours': 10000, '_log_step_count_steps': 100, '_train_distribute': None, '_device_fn': None, '_service': None, '_cluster_spec': <tensorflow.python.training.server_lib.ClusterSpec object at 0x7fcc665c3208>, '_task_type': 'worker', '_task_id': 0, '_global_id_in_cluster': 0, '_master': '', '_evaluation_master': '', '_is_chief': True, '_num_ps_replicas': 0, '_num_worker_replicas': 1}\n"
     ]
    }
   ],
   "source": [
    "model = tf.estimator.Estimator(model_fn=model_fn,\n",
    "                               params=params,\n",
    "                               model_dir=\"./checkpoints_tutorial17-2/\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 训练\n",
    "\n",
    "现在我们新的Estimator已经被创建，我们可以训练它。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Calling model_fn.\n",
      "INFO:tensorflow:Done calling model_fn.\n",
      "INFO:tensorflow:Create CheckpointSaverHook.\n",
      "INFO:tensorflow:Graph was finalized.\n",
      "INFO:tensorflow:Running local_init_op.\n",
      "INFO:tensorflow:Done running local_init_op.\n",
      "INFO:tensorflow:Saving checkpoints for 0 into ./checkpoints_tutorial17-2/model.ckpt.\n",
      "INFO:tensorflow:loss = 2.328683303358867, step = 0\n",
      "INFO:tensorflow:global_step/sec: 30.0746\n",
      "INFO:tensorflow:loss = 1.0425889833487076, step = 100 (3.326 sec)\n",
      "INFO:tensorflow:global_step/sec: 30.7697\n",
      "INFO:tensorflow:loss = 0.4519329631053862, step = 200 (3.250 sec)\n",
      "INFO:tensorflow:global_step/sec: 30.5945\n",
      "INFO:tensorflow:loss = 0.28173916577119856, step = 300 (3.269 sec)\n",
      "INFO:tensorflow:global_step/sec: 30.3772\n",
      "INFO:tensorflow:loss = 0.41579200542133726, step = 400 (3.292 sec)\n",
      "INFO:tensorflow:global_step/sec: 31.44\n",
      "INFO:tensorflow:loss = 0.2537537261934676, step = 500 (3.181 sec)\n",
      "INFO:tensorflow:global_step/sec: 32.2734\n",
      "INFO:tensorflow:loss = 0.2306796091927107, step = 600 (3.103 sec)\n",
      "INFO:tensorflow:global_step/sec: 32.4727\n",
      "INFO:tensorflow:loss = 0.16169791614095563, step = 700 (3.075 sec)\n",
      "INFO:tensorflow:global_step/sec: 32.9575\n",
      "INFO:tensorflow:loss = 0.24491770370504626, step = 800 (3.034 sec)\n",
      "INFO:tensorflow:global_step/sec: 31.4056\n",
      "INFO:tensorflow:loss = 0.1723769961825516, step = 900 (3.185 sec)\n",
      "INFO:tensorflow:global_step/sec: 31.8268\n",
      "INFO:tensorflow:loss = 0.0865023047044578, step = 1000 (3.142 sec)\n",
      "INFO:tensorflow:global_step/sec: 33.1043\n",
      "INFO:tensorflow:loss = 0.08865380930537742, step = 1100 (3.021 sec)\n",
      "INFO:tensorflow:global_step/sec: 33.0132\n",
      "INFO:tensorflow:loss = 0.09500106271291871, step = 1200 (3.029 sec)\n",
      "INFO:tensorflow:global_step/sec: 32.2879\n",
      "INFO:tensorflow:loss = 0.048251991971276796, step = 1300 (3.097 sec)\n",
      "INFO:tensorflow:global_step/sec: 32.4468\n",
      "INFO:tensorflow:loss = 0.0965478484811222, step = 1400 (3.082 sec)\n",
      "INFO:tensorflow:global_step/sec: 31.0871\n",
      "INFO:tensorflow:loss = 0.06810141978839185, step = 1500 (3.217 sec)\n",
      "INFO:tensorflow:global_step/sec: 31.6667\n",
      "INFO:tensorflow:loss = 0.13537004696386645, step = 1600 (3.158 sec)\n",
      "INFO:tensorflow:global_step/sec: 31.98\n",
      "INFO:tensorflow:loss = 0.08716099232839157, step = 1700 (3.127 sec)\n",
      "INFO:tensorflow:global_step/sec: 32.1884\n",
      "INFO:tensorflow:loss = 0.06138957874514458, step = 1800 (3.107 sec)\n",
      "INFO:tensorflow:global_step/sec: 32.1328\n",
      "INFO:tensorflow:loss = 0.11381113679326431, step = 1900 (3.113 sec)\n",
      "INFO:tensorflow:Saving checkpoints for 2000 into ./checkpoints_tutorial17-2/model.ckpt.\n",
      "INFO:tensorflow:Loss for final step: 0.09910375161965862.\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<tensorflow.python.estimator.estimator.Estimator at 0x7fcc665c3a58>"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.train(input_fn=train_input_fn, steps=2000)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 评估\n",
    "\n",
    "模型一旦被训练好，我们可以在测试集上去评估它的性能。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Calling model_fn.\n",
      "INFO:tensorflow:Done calling model_fn.\n",
      "INFO:tensorflow:Starting evaluation at 2018-07-16-11:24:18\n",
      "INFO:tensorflow:Graph was finalized.\n",
      "INFO:tensorflow:Restoring parameters from ./checkpoints_tutorial17-2/model.ckpt-2000\n",
      "INFO:tensorflow:Running local_init_op.\n",
      "INFO:tensorflow:Done running local_init_op.\n",
      "INFO:tensorflow:Finished evaluation at 2018-07-16-11:24:20\n",
      "INFO:tensorflow:Saving dict for global step 2000: accuracy = 0.9769, global_step = 2000, loss = 0.0701695\n",
      "INFO:tensorflow:Saving 'checkpoint_path' summary for global step 2000: ./checkpoints_tutorial17-2/model.ckpt-2000\n"
     ]
    }
   ],
   "source": [
    "result = model.evaluate(input_fn=test_input_fn)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'accuracy': 0.9769, 'global_step': 2000, 'loss': 0.0701695}"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "result"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Classification accuracy: 97.69%\n"
     ]
    }
   ],
   "source": [
    "print(\"Classification accuracy: {0:.2%}\".format(result[\"accuracy\"]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 预测\n",
    "\n",
    "模型也可以用来在新的数据上进行预测"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "predictions = model.predict(input_fn=predict_input_fn)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Calling model_fn.\n",
      "INFO:tensorflow:Done calling model_fn.\n",
      "INFO:tensorflow:Graph was finalized.\n",
      "INFO:tensorflow:Restoring parameters from ./checkpoints_tutorial17-2/model.ckpt-2000\n",
      "INFO:tensorflow:Running local_init_op.\n",
      "INFO:tensorflow:Done running local_init_op.\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "array([7, 2, 1, 0, 4, 1, 4, 9, 5])"
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cls_pred = np.array(list(predictions))\n",
    "cls_pred"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAU0AAAD5CAYAAACj3GcTAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xe8VNW5//HPg2IQVBCxU86NWECjmGCvV5EoFpQYxViIsUQxlngDSYwSTJAYbPwsV1FfoglWEBGJimJXQISASrGAokEvIqJGiWJ7fn/MXjN7Tp19ph++79eL19kzs8uj68w6z1p77bXM3RERkdy0KncAIiLVRJWmiEgCqjRFRBJQpSkikoAqTRGRBFRpiogkoEpTRCQBVZoiIgmo0hQRSWDdfA7u1KmT19TUFCiU6jBnzpyV7r5pueMoFZVxy6cyTiavSrOmpobZs2fnc4qqY2bvlDuGUlIZt3wq42TUPBcRSUCVpohIAqo0RUQSUKUpIpKAKk0RkQTyunsu0lxXXnklAF988QUAr7zyCgATJkyos+/ZZ58NwF577QXAySefXIoQReqlTFNEJAFlmlJSxx9/PADjx4+v93Mzq/PeTTfdBMC0adMAOOCAAwDo2rVrMUKUMnrjjTcA2H777QG49tprATj33HPLFlNtyjRFRBJQpilFF7JLaDjD3GGHHQA49NBDAXjrrbfSn02ePBmAxYsXAzBu3DgALrroosIHK2U1d+5cAFq1SuVzW2+9dTnDqZcyTRGRBJRpStGE55kfeOCBOp/ttNNOQCaL7NSpEwAbbLABAF999VV63z322AOAl19+GYCPPvqoSBFLuc2bNw/I/B4MGDCgnOHUS5mmiEgCJc80wzi8W265BYCtttoq/VmbNm0AOPHEEwHYYostAOjevXspQ5QC+b//+z8A3D39Xsgwp06dCsCWW25Z77FhHCfAokWLsj474ogjChqnlN+rr74KwHXXXQfAKaecUs5wGqVMU0QkgZJnmkOGDAFg6dKlDe4TxuVttNFGAPTs2bMg1+7SpQsAQ4cOBaB3794FOa/U78gjjwQyd70BNtxwQwA6duzY6LH33ntvejvevykt0+uvvw7A6tWrgewRF5VGmaaISAKqNEVEEih58/zWW28FMsNH4k3vhQsXApkBrk8//TQAM2fOBDKPzb377rsNnr9169ZAZghLuBkRP09opqt5XhrdunXLed8rrrgCyDxOFxeGHoWf0nKMGjUKSC29AZX93VSmKSKSQMkzzYMPPjjrZ1x4hC74+OOPgUzmGf76vPTSSw2e/3vf+x6QeeA/PJ4HsGrVKgC22WabZsUuxTNlyhQAhg0bBsCaNWvSn22++eYAXH755QC0bdu2xNFJMcRvBofvdPjetmvXrhwh5USZpohIAhX9GOXGG28MwEEHHZT1fn1Zam33338/kMlWAXbeeWcABg4cWKgQpUDCI5fxDDMIw0/ClHDSMjzzzDN13tt008pfbl6ZpohIAhWdaTbHihUrABg8eDCQ/Qhf6C9ramC1lM7RRx8NZB6rDAYNGpTeHjFiREljktIIS5zEhQdPKpkyTRGRBFpcpnnDDTcAmYyzQ4cO6c/CnTkpvzB+dvr06UCmLzP0aV188cXpfcM0YdIyzJgxA4CxY8em39t1110BOOSQQ8oSUxLKNEVEEmgxmebzzz8PZMbyBQ8++GB6O0xLJuUXJpdduXJl1vthWkCNpW25nnjiCSB7ZEsYox2mh6xkyjRFRBJQpSkikkCLaZ4//PDDQGbuxT59+gCw1157lS0mqSusCRQejQ0OPPBAAP70pz+VOiQpsTBZT9xPf/rTMkTSPMo0RUQSqPpM84svvgDg0UcfBTITdlx66aVAZqo4KZ/46pEjR44E6s7G3qtXL0DDi1qy5cuXA/Dcc88B2ZPpHHPMMWWJqTmUaYqIJFD1mWaYtDb0kR122GEA7L333mWLSbJdddVV6e1Zs2ZlfRYeo1RfZst3++23A/DBBx8Ame9qtVGmKSKSQFVmmmHCWoA///nPALRv3x6ASy65pCwxScOuvvrqBj8Lj72qL7Ple+edd7Jeh6kfq40yTRGRBKoq0wx3Yc8777z0e9988w0A/fr1AzQus9qEMs1llENoTYR9v/76awA+/fTTOvuGR/Suueaaes+1zjrrpLf/+te/AlpGo9geeuihrNdHHHFEmSLJjzJNEZEEVGmKiCRQFc3zb7/9FsjMhPL222+nP+vevTuQuSEk1SWs25SL4447DoAtt9wSyAxdueeee/KKIax2GZ/DUwonDGYP5VXtlGmKiCRQFZnmkiVLgMyKhXFhOIvmX6xc4SYdwKRJk5p9nvvuu6/JfcJNolatsvOBo446CoDevXvXOWbfffdtdkzStAceeADI3LQNs7RX6+qiyjRFRBKo6EwzDIbt27dv1vtXXnllertahy2sTSZOnJjeHjVqFFB3wo5g4cKFQOP9lKeddhoA3bp1q/PZT37yEwB69OjRvGClYP7zn/8A8Mgjj2S9H6aBiw/7qibKNEVEEqjoTHPMmDFA3cev4n0hZlbSmCQ/ua5rfddddxU5Eim20L8cVoTt378/AOeff37ZYioEZZoiIglUZKYZxnVdf/31ZY5ERJorZJphnfOWQpmmiEgCFZlphjXMP/vss6z3w9M/mkZMRMpFmaaISAKqNEVEEqjI5nltYaXCJ554AoCOHTuWMxwRWYsp0xQRSaAiM83f//73WT9FRCqFMk0RkQTM3Zt/sNmHwDtN7tiydHP3TcsdRKmojFs+lXEyeVWaIiJrGzXPRUQSUKUpIpJAo5WmmW1iZvOif8vN7L3Y6/WKEZCZ9YxdY56ZfWZmv2rimNPN7MNo/0Vm9os8YxhnZkc3sc/vYjEuMLNvzKx9PtcthzKVcTcze9rMFkb/7xot3+iYcpTxKWb2qpm9YmYvmNkP8rlmuZSjjKPr3hHKLMf9y1HGO5rZDDNbY2YX5HRid8/pHzAc+E097xvQKtfzJPkHtAZWAJ2b2O90YHS0vQWwEuhUa591E1x3HHB0gv2PAR4rxv+DUv4rVRkDWwG9ou2NgCXAdpVWxsA+QIdo+0jghXKXUbWUcXTOA4DdgXk57l+OMt4c6A1cDlyQy3mb1Tw3s+5RlnAnsADoYmafxD4faGa3Rtubm9lEM5ttZrPMbM8ElzoEWOTuy3I9wN2XA0uBrmY2wsz+ZmYvALeb2bpmdnUUxytmdnoUYysz+18ze83MHgc6JYgR4ATg7oTHVLRilrG7v+/u86LtfwOvAVvnGlupytjdX3D38N88E+ica4zVoNjfY3d/BljVnNhKWMYfuPts4JtcY8tncPsOwCnuPtvMGjvPtcAod59pZjXAFGAnM9sDONXdz2rk2IEkrIzMrDvQDXgrFuf+7v6lmQ0GVrj77mb2PWCmmT0G7An8F9CTVBa0ELgpOt9lpDKMhxu43gZAH+CMJHFWiaKXsZl9H9gJeCnXoEpdxpHTgEca+bxaleJ7nFiZyjgn+VSaS6Iauil9gO0tsyzFxma2vru/CLzY0EFm1gY4HLgwx3hONLMDgTXA6e7+SXTNB939y2ifvkAPMxsYvW4PbAvsD9zt7t8By8zs6XBSd/9DE9ftDzzj7p/mGGc1KXYZbwTcD5zr7p/ncJ2ylLGZ9QFOBlriWr9FLeNmKNf3OGf5VJqrY9vfkeoTCdrEtg3Y3d3rX36wYYcDL7r7yhz3v9Pd6+vIjcdpwGB3fyK+g5kdkzC2uIHA3/M4vpIVrYwtdQNiIjDW3SfneFjJy9jMegFjgB+7+8fNOUeFK/b3OKlyfY9zVpAhR1HN/rGZbWtmrUjdGAmmAeeEF9EvYS7q9BOa2flmlk8zYCowODRDzGx7M1sfeBY4PuoT2ZpUB3aTzGxjYG/goTxiqgqFLGNLpQ63k7pBcG2tzyqmjKNm6ATgZ+6+OI+YqkKRvsd1VFIZN0chx2n+ltR/zHQgfuPmHGCfqMN2IVHfn5ntYWY31XciM9sQ+G9gUq2PegAf5RHjGOBNYJ6ZzQduJJVtTwDeJdUHMhZIL2piZpeZWb8GzvcT4BF3/yKPmKpJocr4AFJ/FA+xzNCXH0efVVIZDwc6AmOiGAvZDK1UhfwejweeA3qa2TIz+3n0UcWUsZl1NrNlwHnA8CjOto1dvKoeozSzfwD93T3nO11SXVTGLV+1l3FVVZoiIuWmxyhFRBJQpSkikoAqTRGRBFRpiogkkNcaQZ06dfKampoChVId5syZs9LXolm9VcYtn8o4mbwqzZqaGmbPzuUJrJbDzNaqZQFUxi2fyjgZNc9FRBJQpSkikoAqTRGRBFRpiogkoEpTRCQBVZoiIgnkNeSoVFavTs0/OmTIEABuuikzE1Xv3r0BGD9+PADdunUrcXQisjZRpikikkBVZJrvv/8+ALfccgsA66yzTvqzMCj3oYdSk6f/6ldNLqEtFeCf//wnAAMGDABg6dKlzT7XY489lt7u0aMHAF26dGl+cFI24Xt81FFHAXDdddcBcPbZZ6f3iX//y0GZpohIAhWdaX744YcADBo0qMyRSKFNnToVgDVr1uR9rsmTM+uy3XbbbQDcc889eZ9XSuejj1KrX8QzSoBzzz0XgNNOOy393vrrr1+6wOqhTFNEJIGKzDSvvTa1QOGkSal11V566aUmj3nuuecACMt37LLLLgDsv//+xQhRmumbb1LLwjz88MMFO2cYQQFw9dVXA5kRF+3atSvYdaR4nn32WQDee++9rPdPOOEEANq0aVPnmHJRpikikkBFZpoXXJBaKz7JXbKJEydm/ezatSsA9913X3qfH/3oR4UKUZrpqaeeAmD69OkA/Pa3v837nKtWrUpvL1iwAID//Oc/gDLNShbvzx4xYkS9+5x88skAmFlJYsqFMk0RkQRUaYqIJFBRzfN+/foBmZs53377bZPHdOrUCcg0w955JzUh89tvvw3Abrvtlt73u+++K1ywkrNXX301vT1w4EAAunfvDsBFF12U9/njQ46kerzyyivp7fCwQ7Duuqmq6bDDDitpTLlQpikikkDZM81nnnkmvf3aa68BmU7fhm4EnXXWWentvn37AtC+fXsAnnzySQAuu+yyOsfdeOONQN0BtFJc8bIIN2jGjRsHwAYbbNDs84YbQPHfoUq6YSCNCzdt63PIIYeUMJJklGmKiCRQtkwzTNAQ+rgAVq5cWe++YfjQscceC8Af//jH9Gdt27bN2jdMDTdmzJg65xw6dCgAX375JZCZ3KN169bN+4+QRk2YMAHIHsge+jLjfc3NFYapxLPLAw88EIAOHTrkfX4prngLIVhvvfUAGDlyZKnDyZkyTRGRBMqWaX799ddAw9klZB6BvPfee4HMnfLGhEwz3JW98MIL05+FR+tCxhmmn9pmm20SxS65CRNDh//vUJj+5NBKueuuu4DMnVaAiy++GFDroZKFBxtmzJhR57PQcuzVq1dJY0pCmaaISAJlv3ten9DfNXbsWCC3DLO2kEXeeeed6fdmzZpVgOikKZ9++ikAM2fOrPPZ4MGD8z7/zTffDGSmDuzZs2f6s4MOOijv80txNTYBTzWMbFGmKSKSQNkzzfqe+nnxxRfzPm94qij+FFDtJ43CXfgwZlAKI0zEsGzZMiAzvVehLFmyJOv1TjvtVNDzS3HVl2mG0Q6FaIkUmzJNEZEEVGmKiCRQtuZ5WLu8WCvLhVXt5s6dm36v9uOZl156aVGuvbbbcMMNgcywkfiEHeHRx44dOyY+74oVK4DMUKZgn332aVacUlrPP/88kBkqFhceg+7cuXNJY2oOZZoiIgmULdOcMmVKQc8Xhp8sXLgQaPwxrDCESQOgiyOsFhgemQyPUwIcfvjhQPZDB/WZP39+ejvc+AnT/tWelKNVK/3trwZhxclwQzaukifoqE2/bSIiCZR9yFGhhOnHbrjhhgb3qampAeCOO+4AMhOBSHEMHz4cyM4sQgsjPlFLfTbddNP0dsgsG3rk9tRTT80nTCmR2n3R8UlVzjzzzFKH02zKNEVEEqj6TDMskREmMG5MeNxuv/32K2pMktKjRw8ge0XQMJqh9gD12sI0gHGDBg0C6j6MEPpQpTKFhxxq3zWP3ykvxFSBpaJMU0QkgbJlmo0tnvbII49kvT7jjDMAeP/99xs8Ty7LHBT6jr0kt+uuu2b9TOL73/9+ve/Hx4H+4Ac/aF5gUjRhKrjad8379+9fjnDypkxTRCQBVZoiIgmUrXke5s0Ls6jHhQHQtR+xrO+Ry9C8z2XlSqluoXlXu5mnJnllC4Pag/BwyQUXXFCOcPKmTFNEJIGyZZoDBgwAYNSoUen3GlsvqCnhr1cY5nLLLbcAsOWWWzb7nFJZws0+rW1eXaZOnZr1ukuXLkBmko5qo0xTRCSBsmWaYdXIsNIkwKRJkwAYPXp04vP94Q9/ADJrmUvLE9arDzSovbKFFWcXL16c9X6bNm2A6p0wR5mmiEgCZX+MMqxtHt/u27cvkFl1MEwofOSRRwLwy1/+Mn1MuJMaX5FQWqawOmmY6GHYsGHlDEeaEKbsC49ILliwAIBtt922bDEVgjJNEZEEyp5p1ufQQw/N+ikCmYzl17/+NaA1zitdGDsdpm0Mox5++MMfli2mQlCmKSKSQEVmmiL1CX3bUl222morAG677bYyR1IYyjRFRBJQpSkikoAqTRGRBFRpiogkoEpTRCQBVZoiIglY7QldEx1s9iHwTuHCqQrd3H3TpndrGVTGLZ/KOJm8Kk0RkbWNmuciIgmo0hQRSUCVpohIAo1Wmma2iZnNi/4tN7P3Yq/XK1ZQZtbPzF43s8VmNiSH/UfEYnvVzA7P8/rPm1mvJvZpY2YTohhnmFnXfK5ZLuUq4+ja65rZK2Y2KYd9S17GsX2PNzPPdf9KU8bv8R1m9qGZzctx/9PD/ma2yMx+kef1x5nZ0U3ss2P0/V1jZjktj9nohB3u/hHQKzr5cOBzd7+y1kWN1A2l73K5YFPMrDVwPfDfwHJgtpk96O5vNHHoFe4+2sx2Ap4ys808dpfLzNZ1928KEWPkTGC5u3c3s5OAvwAnFvD8JVGOMo65EJgPtM1x/1KXMWa2ETAYmF3I85ZSGcv4NuAG4OYEx9zp7heY2RbAfDOb7O7pFReLUMYrgXOBY3M9oFnNczPrbmYLzexOYAHQxcw+iX0+0MxujbY3N7OJZjbbzGaZ2Z5NnH5PYJG7v+Pua4D7gP65xubu8wEDNo7+0txoZrOAkWa2gZndHsUx18yOjGJsa2bjo79u9wNtcrhUf+COaPs+4Me5xlgNilzGmFk34BBgbNLYSljGACOjf2uSxlnpil3G7v4MsKo5sbn7cmAp0DVqZfzNzF4Abo9aKFdHcbxiZqdHMbYys/81s9fM7HGgUw7X+cDdZwM5V8T59GnuAFzj7j2B9xrZ71pglLv3Bo4DQiHsYWY31bP/1sC/Yq+XRe/lxMz2Br5091BYWwJ7uvtQYBjwqLvvDhwEXGVmbYBfAR+7ew9gBLBr7HxjG2iWpeN096+A1WbWIdc4q0SxyhhgNDAESDzmrVRlbGa7AZu5+9Tan7UgxSzjZjOz7kA34K1YnAe7+0mkWnkrojLeDTjHUt1jxwL/BfQETgX2jp3vMjPrV4jY8plPc0lUQzelD7C9Zdaq3tjM1nf3F4EX87h+bUPM7OfAZ8DxsffHx5ocfYHDzOx30es2QFdgf2AUgLvPNbMF4WB3P7WAMVabopRx1M/0L3efZ2Z9EsRTsjI2s1bAVVRhl0tClfY9PtHMDiSV2Z/u7p9E13zQ3cNypH2BHmY2MHrdHtiWVBnfHf0uLDOzp8NJ3f0PhQown0pzdWz7O1LNpSDe9DFg9ygby8V7QJfY6840/hcwuMLd61v7Nx6nAUe7+5L4DrFfhCRCnMst1Znezt0/aeKYalOsMt4bGGBmR0Xn2cjM7nD3QU0cV8oy7kAqY3kuOnYL4GEzO9zd5yY9WQUrVhk3153uXt8NmdplPNjdn4jvYGbHFDWySEGGHEU1+8dmtm30Fzoe/DTgnPCigaZu3Eygp5l1M7PvkWoKTI6OHRX6qJppKqlO3xBLaKI9C/wsem8XYMcczjUZCF/y44DH8oir4hWyjN19qLt3dvca4CTgsVBhVkoZu/sqd+/k7jVRnLOBfi2swsxS4O9xg8zsfDM7q/mRMhUYbGbrRufb3szWJ1XGx0d9m1sDB+RxjQYVcpzmb0n9x0wn1Q8ZnAPsE3XYLgTOgIb7Qtz9a+A84HFgITDO3V+PPt6Z1B315roUaGepISsLgOHR+9cDm5jZIuASIP3FaKRP82ZgSzNbTKq/7KI84qoWBSnjJlRSGa+NClbGZjYeeI5UErQs6loB6AF8lEeMY4A3gXlmNh+4kVSreQLwLql6YywwIxZLvX2aZtbZzJaRqnOGR3E2Opqjap49t1Qb6RF31xKVLZTKeO1gZv8A+hd6eFipVE2lKSJSCfQYpYhIAqo0RUQSUKUpIpJAPuM06dSpk9fU1BQolOowZ86clWvTrN4q45ZPZZxMXpVmTU0Ns2dX7TwGzWJma9WyACrjlk9lnIya5yIiCajSFBFJQJWmiEgCqjRFRBJQpSkikoAqTRGRBFRpiogkkNc4TRGRYvj4448BePfddxvcp1u3bgBcc801AOy0004AbLfddgDssssuRYlNmaaISAIVlWmuWLECgOOOOw6AvfdOrYt05plnAqknFwrh008/BeDZZ58F4NBDU9M3tm7duiDnF5FkpkyZAsBDDz0EwNNPPw3Am2++2eAx22+/PQBLly4FYM2a7AVDv/uu0CtOpyjTFBFJoOyZZui7ANhxx9SyLSET3HzzzYHCZ5g//OEPAVi5MrUGfXjudtttty3IdSR3//73vwH43e9Si0cuWJBaJHLatGnpfdQCaBmWLEmtdXfDDTcAcPPNN6c/++KLLwBIMin666+/3vRORaBMU0QkgbJlmiHLC/2XAB99lFpr6ZxzUoveXXfddQW95ogRIwB4++23gcxfOmWYpTdu3DgALr74YqDuXdKQgQJssskmpQtMimbZstQ6baNH17cKc+522GEHIHO3vNSUaYqIJFC2TPOf//wnkLlLFjds2LCCXWf+/Pnp7SuvvBKAY45JLed8/PHHF+w6kpuQbfz6178GMi2O1EKUGeeem166nOuvvx6Ajh07liJEaYZQjpDJJPfdd18gMzplvfXWA6B9+/YAbLDBBuljPv/8cwB+/OMfA5ksco899gBg1113Te+7/vrrA9CuXbsC/1fkRpmmiEgCqjRFRBIoefM8DGC///7763x22223AbDppvkvzxKa5YccckidzwYMGADAhhtumPd1JJnQRRJu+jXknnvuSW8/8sgjQOamUWi6h+aelM/q1auB7O/Zyy+/DMCkSZOy9t1rr70AmDt3LpA9lDDcCOzcuTMArVpVbj5XuZGJiFSgkmea//M//wNkhpyEgeYAP/3pTwt2neeffx6A5cuXp9879dRTATjppJMKdh1p2jvvZNawGjt2bNZnYVKF8CDD448/Xuf48FBCyFJPPPFEALbYYovCBys5+eqrrwD42c9+BmSyS4CLLroIgD59+tR7bH0Pq3Tt2rXAERaPMk0RkQRKnmmGoSXh59Zbb53+LJ8+qvAY1siRI4HMo1rxoSyhz1RKa968eentMGh9//33B+CZZ54B4MsvvwTgrrvuAuAvf/lL+pjFixcDmVZD//79gUxfp4YilU4YGhS+Z2GCjfh9iCFDhgDQtm3bEkdXGso0RUQSKPuEHWFKKIC+ffsC0KFDBwDOPvvsJo8Pg+PDz5kzZ2Z9Xsh+Umme+JRdIfMPg9uDNm3aAPCLX/wCgAkTJqQ/CxM9hMkcQgaju+elF+6IX3755UBmIuDnnnsuvU8YvN5SKdMUEUmg5Jnm+eefD8CTTz4JwPvvv5/+LPRvhYziwQcfbPJ8Yd/aj+Fts802QKbvRcrn7rvvrvPeP/7xDwCOPvroeo8J0/XVZ8899wSyH8OT0pg+fXrW6/B4YxhfuTZQpikikkDJM80f/ehHALz66qtA9p3VRx99FIBRo0YBsNlmmwEwaNCgBs938sknA7DzzjtnvR+WyggZp5TPCSeckN4OrYeXXnoJgNdeew3I/D488MADQPbk1KGPO7wXpvQLZd+zZ8+ixS7Z4n3NkBnBcOmll6bfO+qoo4DsSTZaEmWaIiIJqNIUEUnAkqzJUVvv3r29sQ77UnjrrbeATDO8V69eADz22GNAYSb/iDOzOe7eu6AnrWCFKONVq1alt0M5hUcjG7qRF58AIjyocMQRRwDwxhtvAJlVSm+66aa84qtNZdyw2g+n1GedddYB4KyzzgIyc2L+61//AqB79+5AZk2wuLBGVJjco1g3mPIpY2WaIiIJlH1we77+9Kc/AZm/fOEmUqEzTGm++GOO48ePB+DYY48F6mac5513HgB//etf08eEge9hSr/wiOXUqVOBzOB30I2/YvvNb34DwFVXXdXgPt9++y2QaSGEn0mEm8AHHnggkD1VYLkp0xQRSaAqM82QrQDccccdAGy00UaAVi6sdGG6sDB0JUzQEYYVhZZDyC7jLrnkEgAWLVoEZIYvhWMg8/sgxREenwyryIZp+r7++uv0PmEdqJBxNkeYrDx81+MrT4bJqMtFmaaISAJVmWmGAbVxhx9+OJA9qbFUrpBxNjRRbX3CKoRhFdGQaT711FPpfcKdek0XVxzhzvhuu+0GZEYyxD3xxBNAJvscPnw4ALNmzUp8vdDXPWfOnMTHFosyTRGRBKo+0wxrH4e7etLyhf60yZMnA9l3VsMa6cOGDSt9YALAwQcfnPU6PCodMs3WrVsDmeVnAM444wwArrnmGiDT112JlGmKiCSgSlNEJIGqap6Hx+XiK0yGVQx1A2jtEdbEHjp0KJC9vna46TBw4EAAtttuu9IGJ3WEFRnCKpXhBlGYrQrgzTffBDIrMNQWX0us3JRpiogkUJWZZnyygH79+mXt89lnnwGZuReraT1lSSZMzvLnP/85/V64Ifj73/8egHHjxgGZ4UpSej169AAyQ8XuvffeOvvEh40BrLtuqmoKQwnjj9XY76NRAAAG3klEQVSWmzJNEZEEqirTrE/4ixQyijBkITx2pcfqWr5TTjklvT1mzBgAJk6cCGT6ymrP7C+lE7L80aNHA5nWYHzA+gcffABATU0NkCnT0EddSZRpiogkUPWZ5i233ALArbfeCsDpp58OZCZ3kJYvPg3gtGnTgMx63GGCiUoeLL22CCNdpkyZAsDf//739GczZswAMpllmBquEinTFBFJoKoyzeuuuw6AP/7xj+n39t9/fwDOPvtsADbeeGMA1ltvvRJHJ5UgjJYIy2WERy0XLlwIaOXKShJWE629XemUaYqIJFBVmeZ+++0HwJNPPlnmSKTShUmOd9llFwAWL14MKNOU/CnTFBFJQJWmiEgCVdU8F8lVWDPq7bffLnMk0tIo0xQRSUCVpohIAqo0RUQSsLDaW7MONvsQeKdw4VSFbu6+adO7tQwq45ZPZZxMXpWmiMjaRs1zEZEEVGmKiCTQaKVpZpuY2bzo33Izey/2uqgzYpjZumb2iplNymHfEbHYXjWzw/O89vNm1ivHfY83M891/0pTrjI2swvNbEH079wc9j/dzD6M4lpkZr/I8/rjzOzoJvbpaGaTo9/DF82sKp/BLGMZL4u+j/PM7MUc9i9HGfcxs09j/z/+0NR5Gx3c7u4fAb2ikw8HPnf3K2td1Ej1jX7X1MUSuhCYD7TNcf8r3H20me0EPGVmm3msw9bM1nX3bwoZoJltBAwGZhfyvKVUjjKO/sAMAnoD3wCPmdkUd29qJPqd7n6BmW0BzDezye6+MnbeQpfxJcCL7n6Ume0I/D/gkAKevyTK/D3ez90/SbB/qcsY4Cl3b7RyjWtW89zMupvZQjO7E1gAdDGzT2KfDzSzW6Ptzc1sopnNNrNZZrZnDufvRuqXc2zS2Nx9PmDAxtFfmhvNbBYw0sw2MLPbozjmmtmR0fXamtn46K/b/UCbHC83Mvq3Jmmcla7IZdwDmOnuX7j718CzwDG5xubuy4GlQNeolfE3M3sBuD1qoVwdxfGKmZ0exdjKzP7XzF4zs8eBTjlcqifwZHTNBcB2ZrZJrnFWumJ/j/NRwjJOLJ8+zR2Aa9y9J/BeI/tdC4xy997AcUAohD3M7KYGjhkNDAES39o3s72BL919VfTWlsCe7j4UGAY86u67AwcBV5lZG+BXwMfu3gMYAewaO99Yq6fpbWa7AZu5+9SkMVaRYpXxq8ABlmr+tgMOA7rkGpSZdQe6AW/F4jzY3U8CzgRWRGW8G3COmXUFjgX+i1RFeCqwd+x8l5lZ9rKmKS8DA6J99gI6R/9akmJ+jx140szmmNlpSYIqYRkD7GtmL5vZw5ZDF0w+z54vcfdcmqV9gO0ts+zuxma2vru/CNTp54j6IP7l7vPMrE+CeIaY2c+Bz4DjY++PjzU5+gKHmdnvotdtgK7A/sAoAHefa2YLwsHufmo9MbYCrgJOTBBfNSpKGbv7fDO7GpgGfA7MBb7N4TonmtmBpDL70939k+iaD7r7l9E+fYEeZjYwet0e2JZUGd8d/S4sM7OnY/E01I91GXCtmc0jVYG+nGOc1aQoZRzZ093fi5raj5vZInef3sR1Sl3GLwE17v551PKcSKqCblA+lebq2PZ3pJrEQbx5a8Du7v5VjufdGxhgZkdF59nIzO5w90FNHHeFu49uIk4Djnb3JfEdYr8IuepA6q/Zc9GxWwAPm9nh7j436ckqWLHKGHe/GbgZwMxGAYtzOOxOd7+giTgNGOzuT8R3MLOcm/+xGD8l1fca/lAuBVraDCDFLOP3op/LzexBYHegqUqzHGUcth+KuvM6NNYPW5AhR1HN/rGZbRv9csWDnwacE17U19Stda6h7t7Z3WuAk4DHQoVpZqNCP2QzTQXSd2rNLDTDnwV+Fr23C7BjEzGucvdO7l4TxTkb6NfCKswshSzjaJ/Nop81wFHAPdHr883srDxCnQoMNrN1o/Ntb2brkyrj46N+r62BA3KIsYOZtY5e/hKY5u6rGzummhWyjC11/2CDaLsdqXsU86PXlVTGW8S29wS+aerGVSHHaf6W1H/MdGBZ7P1zgH2iDtuFwBlRgI31hTRkZ2B5HjFeCrSz1DCIBcDw6P3rgU3MbBGpO6bpyq+hPs21VCHLeFK07yTgLHf/d/R+D+CjPGIcA7wJzDOz+cCNpFpUE4B3gYWkbjDOCAc00t/1A2Chmb0OHExqREdLV6gy3hJ4wcxeBmYBD7j7tOizSirjgZYa9jYPuIbsrr16Vc1jlJZqBz/i7oeWOxYpHjP7B9C/CMNKpEJUexlXTaUpIlIJ9BiliEgCqjRFRBJQpSkikoAqTRGRBFRpiogkoEpTRCQBVZoiIgn8f6S1hmwDdPmXAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7fcbfbb44a90>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_images(images=some_images,\n",
    "            cls_true=some_images_cls,\n",
    "            cls_pred=cls_pred)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 总结\n",
    "\n",
    "这个教程展示给我们如何在TensorFlow中去使用Estimator API。它被认为可以使训练和使用模型变得更容易，但它似乎有几个设计问题：\n",
    "\n",
    "* Estimator API是复杂的，不一致的并且有些让人困惑。\n",
    "* 错误报告相当长并且经常难以理解。\n",
    "* 当你想用一个训练好的模型去预测新的数据，每一次 TensorFlow的图都会被再次创建并且加载checkpoint一次。有一些模型非常大所以这可能会会是一个很大的开销。一个更好的方法是，如果检查点在磁盘上发生了变化，则只重新载入模型。\n",
    "* 目前还不清楚如何获得训练好的模型，例如画出神经网络的权重。\n",
    "\n",
    "似乎Estimator API可以更简单，更容易使用。对于小的项目你可能发现它太复杂了并且困惑付出的努力是否值得，但是如果你有一个非常大的数据集，并且如果你在许多机器上进行训练，那么Estiator API是有用的。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 练习\n",
    "\n",
    "下面使一些可能会让你提升TensorFlow技能的一些建议练习。为了学习如何更合适地使用TensorFlow，实践经验是很重要的。\n",
    "\n",
    "在你对这个Notebook进行修改之前，可能需要先备份一下。\n",
    "\n",
    "* 对每个模型运行另10000训练的迭代。\n",
    "* 在优化之前和经过1000，2000，10000次迭代后，打印测试集上印分类的准确率。\n",
    "* 在Estimator里改变神经网络的结构。你需要删除checkpoint文件吗？为什么？\n",
    "* 改变input-functions的batch-size。\n",
    "* 在之前的教程中我们画出被错误分类的图像。请在这也完成。\n",
    "* 改变Estimator，用独热编码标签代替整数类别。\n",
    "* 改变input-functions去加载图像文件来代替numpy数组。\n",
    "* 你能找到一个方法来画出神经网络每一层的权重和输出吗？\n",
    "* 列出5个你喜欢和不喜欢Estimator API的点。你有什么改进的建议？也与你应该给开发者一些建议？\n",
    "* 向朋友解释程序是如何工作的。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## License (MIT)\n",
    "\n",
    "Copyright (c) 2016-2017 by [Magnus Erik Hvass Pedersen](http://www.hvass-labs.org/)\n",
    "\n",
    "Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the \"Software\"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:\n",
    "\n",
    "The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.\n",
    "\n",
    "THE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE."
   ]
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}
